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FOREWORD 


During  the  next  few  years,  critical  decisions  will  be  made 
regarding  the  future  of  the  American  health  care  system.    It  is 
imperative  that  these  decisions  be  based  on  a  substantive  analysis 
of  the  facts.    Accordingly,  the  Office  of  Research,  Evaluation 
and  Planning  is  developing  an  evaluation  strategy  for  all  quality 
assurance  programs  managed  by  DHEW.    The  PSRO  Evaluation  Plan  is 
a  major  component  of  this  effort.    It  has  been  designed  to  provide 
an  objective  assessment  of  the  effectiveness  of  the  PSRO  program. 
The  analyses  that  are  proposed  as  part  of  this  plan  are 
expecially  timely  in  light  of  the  major  decisions  that  will  be 
required  for  the  enactment  of  National  Health  Insurance. 

I  wish  to  express  my  appreciation  to  the  many  persons  who 
were  consulted  and  gave  us  helpful  suggestions. 

I  also  wish  to  commend  Dr.  Martin  Baum  for  directing  the 
development  of  the  plan;  Dr.  Zachary  Dyckman  for  preparing  the 
report  on  baseline  data,  Dr.  Dale  Schumacher  who  prepared  the 
report  on  diagnoses  to  be  used  in  evaluation,  and  Drs.  Allen 
Berkowitz  and  Ron  Lessing  for  their  contributions  to  the 
statistical  design  of  the  evaluation  plan. 


John  O'Rourke 
Di  rector 

Office  of  Research  Evaluation 
and  Planning 
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CHAPTER  I 
BACKGROUND  AND  PURPOSE 


In  1972,  Congress  enacted  the  PSRO  legislation  (PL  92-603)  to  assure 
that  the  services  provided  under  Medicare,  Medicaid  and  Maternal  and 
Child  Health  If  are  medically  necessary,  meet  professionally  recog- 
nized standards,  and  are  provided  in  the  most  appropriate  setting. 

In  Section  1163(e)  and  (f)  of  that  legislation  the  duties  and  report- 
ing requirements  of  the  National  Professional  Standards  Review 
Council  (NPSRC)  are  delineated.    These  duties  include: 

(a)  "review  the  operations  of  . . .  Professional  Standards  Review 
Organizations  with  a  view  to  determining  the  effectiveness 
and  comparative  performance  of  such  organizations;" 

(b)  "make  or  arrange  for  the  making  of  studies  and  investigations 
with  a  view  to  developing  and  recommending  to  the  Secretary 
and  to  the  Congress  measures  designed  more  effectively  to 
accomplish  the  purposes  and  objectives  of  (the  PSRO  program);" 

(c)  "not  less  often  than  annually,  submit  to  the  Secretary  and 
to  the  Congress  a  report  on  . . .  the  findings  of  its  studies 
and  investigations  together  with  any  recommendations  it  may 
have  with  respect  to  the  more  effective  accomplishment  of 
the  purposes  and  objectives  of  (the  PSRO  program).  Such 
report  shall  also  contain  comparative  data  indicating 
results  of  review  activities  conducted  pursuant  to  this 
part,  in  each  State  and  in  each  of  the  various  areas  thereof." 

This  section  of  PL  92-603,  then,  provides  the  legislative  mandate  for 
the  evaluation  of  the  PSRO  program.    In  addition,  evaluation  of  the 
PSRO  program  is  critically  important  because  of: 

(a)  the  substantial  impact  PSROs  can  have  on  Federal  health 
expenditures; 

(b)  its  potential  impact  upon  the  entire  health  care  system, 
in  particular,  as  it  relates  to  the  practice  of  medicine; 


—    Hereafter,  the  term  Medicube  or  the  symbol  MJ  will  be  used  to 
denote  the  three  titles  under  the  Social  Security  Act  that  are  of 
concern  to  the  PSRO  program,  viz,  Title  V,  Maternal  and  Child 
Health;  Title  XVIII,  Medicare;  and  Title  XIX,  Medicaid. 
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(c)  the  need  to  gain  a  better  understanding  of  the  implications 
of  the  different  mechanisms  and  their  variations  for 
quality  assurance;  and 

(d)  the  need  to  establish  an  operational  quality  assurance 
system  prior  to  the  implementation  of  a  National  Health 
Insurance  program. 

The  purpose  of  this  document,  therefore,  is  to  provide  a  detailed  plan 
for  how  the  PSRO  program  will  be  evaluated.    Primarily,  the  document 
if  intended  to  be  used  by  DHEW  for  the  evaluation  and  long  range  manage- 
ment of  the  PSRO  program,  and  by  the  National  Professional  Standards 
Review  Council  to  report  to  the  Secretary  and  to  Congress  on  the  over- 
all effectiveness  of  the  PSRO  program. 

The  primary  focus  of  the  plan  is  the  impact  and  overall  effectiveness 
of  the  program  at  a  regional  and  national  level  of  aggregation.  The 
evaluation  is  not  intended  to  address  explicitly  the  monitoring  of 
individual  PSROs  for  management  purposes. 

The  evaluation  of  a  program  of  this  type  is  extremely  complex.  The 
uneven  implementation  and  enormity  of  the  program,  limitations  in 
measurement  of  human  behavioral  change  and  in  health  outcomes  measure- 
ment technology,  and  the  difficulties  and  expense  in  reconstructing 
utilization,  cost,  and  quality  data  for  the  period  prior  to  the 
inception  of  a  PSRO  are  but  a  few  of  the  complexities.    The  plan  which 
follows  is  therefore  limited  by  the  current  state-of-the-art  in 
evaluation  methodology  and  the  environment  in  which  the  program  is  to 
be  implemented.    This  evaluation  plan  is  also  limited  to  PSRO 
activities  in  short  term  or  acute  care  institutions.    Plans  for  the 
evaluation  of  ambulatory  and  long  term  care  institutions  will  be 
developed  after  definitive  policies  are  promulgated  for  those 
activities . 

The  specifics  of  the  evaluation  plan  are  contained  in  the  following  four 
chapters.    Chapter  II  provides  a  description  of  the  approach  followed 
in  developing  the  evaluation  plan.    The  issues  considered  and  selected 
for  the  evaluation  of  the  program  and  their  specifications  are  contained 
in  Chapters  III  and  IV  respectively.    Chapter  V  contains  an  implement- 
ation plan  for  the  evaluation.    In  addition,  there  are  two  appendices: 
(1)  the  diagnoses  and  procedures  to  be  used  (Appendix  A);  and  (2) 
baseline  data  collection  options  (Appendix  B) . 


Chapter  II 
Approach 
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CHAPTER  II 
APPROACH 


The  approach  utilized  for  the  development  of  the  evaluation  plan  consisted 
of  seven  steps.    The  steps  were  as  follows: 

enumeration  of  program  goals, 

identification  of  evaluation  issues  that  address  the  goals, 
development  of  a  conceptual  framework  for  the  issues, 
assignment  of  priorities  to  the  issues, 
development  of  a  detailed  description  of  each  issue, 
assessment  of  the  feasibility  of  the  analysis,  and 
development  of  an  implementation  plan  for  the  analyses. 

PSRO  PROGRAM  GOALS 

The  primary  goals  established  by  the  National  Professional  Standards  Review 
Council  form  the  core  of  the  evaluation.    These  goals  were  derived  from 
both  the  stated  and  implied  provisions  of  the  PSRO  legislation  (PL  92-603.) 

The  following  goals,  partitioned  according  to  four  basic  areas  of  program 
impact,  were  accepted  by  the  Council  in  early  1974: 

a.  Goals  related  to  quality  of  service 

1.  To  assure  that  services  for  which  reimbursement  is  claimed 
meet  or  exceed  a  minimum  standard  of  quality  for  health 
services  to  all  beneficiaries  of  the  covered  programs. 

2.  To  assure  that  high  quality  care  is  achieved  through 
proper  use  of  health  care  practitioners  in  terms  of  their 
competence. 

b.  Goals  related  to  resource  allocation 

3.  To  assure  that  beneficiaries  are  admitted  to  suitable  types 
of  health  care  institutions  when  such  admissions  are 
required  by  their  health  problems,  and  for  periods  of  time 
as  are  necessary  for  the  proper  medical  management  of  their 
health  problems. 

4.  To  assure  that  the  health  care  services  received  by 
beneficiaries  are  appropriate  to  their  health  problems. 


(a 
(b 
(c 
(d 
(e 
(f 
(9 


5. 


To  promote  the  efficient  use  of  services,  facilities  and 
personnel  in  the  provision  of  medically  necessary  health 
care. 
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c.    Goals  related  to  program  implementation  and  acceptance 

6.    To  determine  if  the  cost  of  establishing  and  operating 
the  PSRO  program  is  commensurate  with  its  impact  on  the 
utilization  and  quality  of  health  care  services. 


7.  To  assure  reasonably  prompt  PSRO  coverage  of  health  care 
services  reimbursed  under  Titles  V,  XVIII,  and  XIX. 

8.  To  maximize  acceptance  of,  support  for,  and  participation 
in  the  PSRO  program  among  health  care  practitioners,  fiscal 
intermediaries  and  carriers,  and  institutional  providers. 

9.  To  educate  the  public  with  respect  to  the  contribution  of 
the  PSROs  to  assuring  and  improving  the  quality  of  health 
care  services  it  is  receiving. 

10.  To  develop  the  necessary  supporting  mechanisms  for  the 
proper  conduct  of  the  PSRO  program  in  maintaining  the 
confidentiality  of  clinical  data  and  the  physician/ 
patient  relationship. 

d.    Goal  related  to  professional  education 

11.  To  assure  that  the  information  gained  from  PSRO  operations 
is  used  to  bring  about  continuing  improvement  in  the 
performance  of  health  care  personnel  on  both  an  individual 
and  group  basis. 


IDENTIFICATION  OF  EVALUATION  ISSUES 

The  determination  of  the  extent  to  which  overall  program  goals  are 
achieved  is  a  complex  matter  involving  multiple  considerations.  For 
example,  to  achieve  goal  three, 

"To  assure  that  beneficiaries  are  admitted  to  suitable 
types  of  health  care  institutions  when  such  admissions 
are  required  by  their  health  problems,  and  for  periods 
of  time  as  are  necessary  for  the  proper  medical  manage- 
ment of  their  health  problems." 

at  least  two  separate  questions  must  be  addressed,  viz.,  are  (a)  unnecessary 
admissions  and  (b)  excessive  lengths  of  stay  prevented?    Each  of  these 
problems  is  addressed  through  a  number  of  different  mechanisms,  (e.g., 
admission  certification,  continued  stay  review,  medical  care  evaluation 
studies,  etc.) 
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A  separate  evaluation  issue  could  be  devised  for  each  mechanism  and/or 
variation.    Consequently 5  a  step  in  the  evaluation  strategy  was  the 
generation  of  the  major  issues  that  can  be  implicitly  and  explicitly 
derived  from  the  stated  goals.    These  are  described  in  Chapters  III  and 
IV. 


CONCEPTUAL  FRAMEWORK  FOR  EVALUATION 

The  following  conceptual  framework  was  developed  to  organize  the  many 
evaluation  issues.    The  timing  of  the  evaluation  analyses  and  their 
results  were  the  basis  of  the  fourfold  classification: 

1.  Interim  Issues  -  benchmarks  which  can  be  used  to  assess  progress 
toward  final  goals.  Such  achievement  can  be  measured  in  terms 
of  overall  program  status  and  status  of  the  individual  program 
components.  This  category  is  needed  because  the  nature  of  the 
PSRO  program  does  not  allow  final  program  goals  to  be  achieved 
during  its  earliest  phases. 

2.  Formative  Issues  -  those  that  are  concerned  with  "forming"  or 
"reforming"  the  structure  or  functioning  of  the  program's 
elements. 

3.  Summative  Issues  -  issues  which  address  the  final  or  ultimate 
objectives  of  the  program.    For  the  most  part,  they  are  derived 
from  explicit  statements  in  the  enabling  legislation  or  can  be 
inferred  from  the  legislative  documentation.    These  issues  are 
those  that  guide  the  measurement  of  whether  the  program  has 
achieved  the  purposes  for  which  it  was  established. 

4.  Secondary  Effects  Issues  -  Most  large-scale  programs  usually 
have  impacts  -  both  positive  and  negative  -  for  which  the 
program  was  not  specifically  designed.    These  secondary  effects 
issues  are  taken  into  consideration  to  obtain  a  comprehensive 
assessment  of  the  full  impact  of  the  program. 


PRIORITY  SETTING 

The  number  and  complexity  of  the  issues  and  the  likelihood  of  scarce 
resources  for  evaluation  required  that  priorities  be  established  for 
the  issues.    Priorities  were  assigned  on  a  three  point  scale,  A  through 
C  (with  A  being  used  to  designate  the  highest  priority).    The  ratings 
were  based  on  Council  and  staff  judgments  of  each  issue's  relative 
overal 1  importance  to  the  evaluation  of  the  PSRO  program  regardless  of  the 
resources  required,    (e.g.,  usefulness  in  assessing  long-term  program 
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impact  and/or  providing  information  which  could  lead  to  program  improve- 
ment through  changes  in  regulations  or  legislation)    Subsequently  the 
list  of  issues  was  further  revised  to  reflect  resource  constraints.  The 
results  of  the  assignment  are  contained  in  Chapters  III  and  IV. 


SPECIFICATION  FOR  THE  MAJOR  ISSUES 

It  became  evident  that  there  were  a  number  of  pervasive  questions  related 
to  each  of  the  issues.    These  questions  were  as  follows: 

(a)  Into  what  sub-issues  does  the  issue  divide  itself? 

(b)  What  analyses  are  required  to  address  the  issue? 

(c)  For  each  analysis,  what  are  the  rationale,  interpretation, 
limitations,  and  pitfalls? 

(d)  What  management  and/or  policy  decisions  can  be  made  based 
on  the  issue  analysis? 

(e)  What  evaluation  measures  need  to  be  formulated  to  perform 
the  required  analyses? 

(f)  What  data  are  needed  to  obtain  the  measures  and  at  what  level 
of  specificity  should  they  be  collected? 

(g)  What  is  the  appropriate  recording  frequency  for  each  data 
element  and  what  is  its  appropriate  source? 

(h)  What  research  is  required  before  the  analyses  can  be  performed? 


Answering  these  questions  yields  a  description  of  each  major  evaluation 
issue  in  sufficient  detail  to  provide: 

(a)  the  basic  information  for  the  preparation  of  requests  for  contract 
(RFC)  to  carry  out  the  recommended  studies  or  analyses; 

(b)  the  basic  guidance  for  performing,  1n  house,  any  of  the 
recommended  evaluation  studies  or  analyses; 

(c)  the  data  requirements  for  the  program  evaluation;  and 

(d)  as  a  by  product,  useful  analyses  for  the  management  of  the 
program. 

Chapter  IV  contains  these  issue  descriptions. 


FEASIBILITY  OF  THE  ANALYSES 

Up  to  this  point,  the  approach  was  mostly  concerned  with  what  should  be  done 
with  respect  to  evaluation.    It  was  also  appropriate  to  investigate  what 
could  be  done.    In  this  regard  there  were  a  number  of  discrepancies  between 
the  should  and  could  of  the  PSRO  program  evaluation.    Some  of  the  reasons 
for  these  discrepancies  are: 
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(a)  The  methodological  state-of-the-art  is  not  far  enough  advanced 
to  provide  the  tools  necessary  to  answer  some  of  the  questions 
posed. 

(b)  Some  analyses  would  not  be  worth  the  cost. 

(c)  Certain  baseline  data  are  unavailable,  unreliable,  invalid. 

(d)  Certain  data  are  too  expensive  to  collect. 

To  assess  the  technical  feasibility  problem,  two  factors  were  considered: 
data  and  methodology.    With  respect  to  data,  it  is  the  baseline  data  (that 
data  pertaining  to  the  period  of  time  prior  to  the  implementation  of  a 
PSRO),  that  is  the  potentially  constraining  element  on  the  feasibility 
of  analysis.    The  collection  of  these  data  is  required  to  evaluate  the 
impact  over  time  of  the  PSRO  program  on  physician  and  institution  behavior 
as  it  relates  to  quality,  utilization,  and  cost.    Data  for  the  period  of 
time  during  the  operation  of  a  PSRO  will  be  available  via  the  PSRO  manage- 
ment information  systems  and  through  special  studies. 

Currently,  there  is  no  national  system  or  systems  which  have  collected 
baseline  data.    Consequently,  it  was  necessary  to  determine  the  feasibility 
and  methods  for  collecting  these  data.    The  technical  feasibility  of 
collecting  baseline  data  was  assessed  by  a  study  in  which  actual  data  was 
identified  and  collected  from  hospital  records.    Appendix  B  contains  a 
description  of  these  data  and  an  analysis  of  the  options  available  for 
their  collection.    No  options  are  recommended  because  additional  information 
regarding  some  of  the  options  are  still  forthcoming.    In  this  regard, 
the  Institute  of  Medicine  of  the  National  Academy  of  Science,  under 
contract  to  DHEW,  is  currently  conducting  a  study  to  determine  whether 
utilization  data  obtained  from  commercial  abstracting  services  can  be 
used  as  a  baseline  for  assessing  the  impact  of  the  program.    As  a  by 
product  of  the  study,  a  national  sample  of  hospitals  (which  could  be  used 
for  the  collection  of  baseline  data)  and  a  more  accurate  estimate  of  the 
cost  to  implement  the  options  discussed  in  Appendix  B  will  be  developed. 
The  results  of  this  study  are  expected  by  January  1976.    At  that  time  the 
exact  method  for  acquiring  the  baseline  data  needed  by  the  evaluation  will 
be  determined. 

Technical  feasibility  in  terms  of  methodology  was  determined  during  the 
development  of  the  issue  descriptions.    These  questions  dealing  with  the 
analysis  were  addressed  in  detail,  so  that  deficiencies  in  methodology 
became  apparent. 

Reductions  in  the  cost  of  performing  the  analyses  can  be  achieved  through 
the  use  of  sampling  techniques.    It  is  presumed  in  sampling  that  if  the 
correct  amount  of  data  is  collected  in  the  right  fashion  that  they  will 
yield  measurements  which  are  an  adequate  approximation  of  the  entire  data 
set. 


8 


There  are  a  number  of  ways  that  sampling  can  be  performed  in  the  evaluation. 
One  method  would  be  to  collect  100%  of  the  data  from  a  selected  set  of 
PSROs  representative  of  various  PSRO  organizational  and  functional  models. 
A  second  method  would  be  to  collect  only  a  portion  of  the  data  for  each 
data  element  from  all  of  the  PSROs.    A  third  method  would  be  some 
combination  of  the  first  two.    For  example,  a  certain  set  of  data  elements 
could  be  collected  from  all  of  the  PSROs  and  another  set  of  data  elements 
collected  from  only  a  sample  of  the  PSROs. 


IMPLEMENTATION 

The  concluding  step  in  the  development  of  the  evaluation  plan  was  the 
consideration  of  implementing  the  plan.    Chapter  V  contains  an  imple- 
mentation strategy  which  was  developed  to  assure  that  the  evaluation  will 
be  performed  successfully,  in  a  timely  fashion,  and  within  overall 
budgetary  constraints.    Accordingly,  the  following  items  were  addressed: 

1.  agencies  responsible  for  implementation, 

2.  time  scheduling,  and 

3.  cost  and  manpower  requirements. 
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CHAPTER  III 
EVALUATION  ISSUES 


The  major  evaluation  issues  that  were  derived  from  the  program  goals 
are  listed  according  to  issue  classification  (interim,  formative, 
summative,  secondary  effects)  in  Tables  1-4  .    These  tables  also 
include  the  sub-issues  and  priority  (as  designated  by  the  NPSRC  and 
OPSR  staff)  for  each  issue.    Because  of  the  large  number  of  important 
evaluation  issues,  the  even  larger  number  of  sub-issues  required  to 
address  each  issue,  and  the  complexity  of  the  analyses,  it  was 
decided  that  only  those  with  the  highest  priority  would  be  considered 
at  this  time  in  the  evaluation  plan.    Lower  priority  issues  are 
identified,  but  their  issue  descriptions  have  not  been  developed.  If 
at  a  later  time,  a  lower  priority  issue  receives  a  higher  priority, 
it  can  be  easily  added  to  the  evaluation  plan.    The  descriptions  for 
high  priority  issues  are  contained  in  Chapter  IV. 

Many  of  these  issues,  in  particular,  those  dealing  with  medical  care 
review,  require  diagnostic  and  procedural  specific  analyses.  This 
means  that  clinical  data  will  be  required.    Because  of  the  costs 
associated  with  collecting  and  analyzing  these,  it  was  not  practical 
or  feasible  to  consider  all  possible  diagnoses  or  procedures.  Therefore, 
it  was  necessary  for  the  evaluation  plan  to  specify  the  diagnoses  and 
procedures  to  be  used  in  the  analyses.    Table  5  contains  the  list  of 
the  selected  diagnoses  and  procedures,  and  Appendix  A  describes  the 
selection  methodology  and  discusses  the  technical  considerations 
related  to  the  availability  and  use  of  the  requisite  clinical  data. 
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TABLE  1 

 INTERIM  ISSUES 

ISSUES 


1.    Have  the  PSRO  program  e-lements  (PSROs,  support 
centers,  State  Councils)  been  developed  and 
placed  into  operation  as  planned? 

a.     What  is  the  status  of  the  program  in 
terms  of  coverage  (geographic  and 
beneficiary  population),  basic  implementa 
tion  schedule  and  cost? 


b.     How  far  advanced  is  the  PSRO  program  in 
terms  of  the  progress  that  has  been  made 
toward  implementing  the  PSRO  functions 
(e.g.,  AC,  CSR,  MCE,  Profiles,  etc.)? 


1  ] 


TABLE  ?. 
FORMATIVE  ISSUES 


ISSUES 


Medical  Care  Review 


PRIORITIES 


1.  Have  admission  certification  and  continued  stay 
review  (concurrent  review)  been  effective  methods 
of  reducing  medically  unnecessary  hospital 
utilization? 

a.  To  what  extent  has  admission  certification 
(AC)  resulted  in  reducing  unnecessary 
admissions? 

b.  To  what  extent  has  continued  stay  review 
(CSR)  resulted  in  reducing  excessive  lengths 
of  stay? 

c.  How  does  the  effectiveness  of  concurrent 
review  vary  with  respect  to: 

.  PSROs 
.  institutions 
.    AC  models  (e.g.  preadmission  certification!) 
.    CSR  models  (e.g.  alternative  percentile 
cutoffs)? 

d.  To  what  extent  has  CSR  resulted  in  medically 
inappropriate  denials  of  extension? 

e.  What  has  been  the  impact  of  PSRO  induced 
changes  in  hospital  utilization  in  terms  of 
Medicube  expenditures  for  health  care? 

f.  Has  utilization  control,  through  concurrent 
review,  resulted  in  net  savings? 

2.  Is  concurrent  quality  assessment  (CQA),  as 
carried  out  in  the  PSRO  program,  an  effective 
method  of  assuring  the  appropriate  utilization 
of  services  (diagnostic  and  therapeutic 
procedures)? 

a.     To  what  extent  has  CQA  (focused  on  assuring 
compliance  with  "essential"  criteria 
elements)  resulted  in  improvements  in 
patient  outcomes? 
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TABLE  2,  FORMATIVE  ISSUES  (Continued) 


ISSUES 

b. 

To  what  extent  has  CQA  resulted  in  a 
reduction  of  deviations  from  criteria  and 
standards? 

c; 

What  is  the  cost  of  CQA? 

3.     Is  retrospective  health  care  review,  as  carried 
out  in  the  PSRO  program,  an  effective  method 
of  assessing  the  quality  and  appropriateness  of 
the  utilization  of  services? 

A 

a. 

How  effective  are  the  methods  employed  by  the 
PSROs  (including  institutional,  practitioner 
and  patient  profiling)  for  identifying  problem 
areas  that  require  further  assessment  by  MCE 
studies? 

b. 

Are  adequate  methodologies  being  utilized  in  the 
performance  of  MCE  Studies? 

c. 

To  what  extent  have  MCE  studies  effected 
corrective  action? 

d. 

Are  some  MCE  models  more  effective  than  others? 

4.     How  effective  are  the  various  corrective  mechanisms 
applied  to  institutions  or  practitioners  for 
correcting  inappropriate  utilization  of  services 
and/or  improving  the  quality  of  care? 

a. 

What  has  been  the  incidence  of  use  of  various 
corrective  mechanisms  relating  to  concurrent 

review? 

b. 

What  has  been  the  impact  of  concurrent  review 
denials  and  special  concurrent  review  require- 
ments (e.g.,  preadmission  certification)  on  the 

incidence  of  inappropriate  utilization  of 
services? 

c. 

Have  recommendations  by  PSROs  for  continuing 
education  had  beneficial  impacts  on 
individual  physician  performance? 

13 


TABLE  2,  FORMATIVE  ISSUES 


(Continued) 


ISSUES 


PRIORITIES 


d.  Have  recommendations  for  organizational  changes 
in  hospitals  (as  a  result  of  PSRO  activities)  le( 
to  a  decrease  in  inappropriate  utilization  of 
services  and/or  improvements  in  quality? 

e.  What  has  been  the  incidence  of  Section  1160  (b) 
sanctions,  and  what  has  been  the  impact  of  such 
sanctions? 


External  Relations 

5.  How  effective  is  the  program  of  communications  for  C 
disseminating  information  concerning  PSRO 

activities  and  achievements  to  institutional 
providers,  professional  societies,  individual 
practitioners  and  medical  educators? 

6.  How  effective  is  the  program  of  communications  C 
for  disseminating  information  concerning  PSRO 

activities  and  achievements  to  the  public? 

7.  What  is  the  cost  of  the  external  relations  C 
program? 


Norms,  Standards,  and  Criteria 

8.     How  appropriate  is  the  variation  among  PSROs  in  B 
terms  of  standards  and  criteria  for  admission 
certification,  continued  stay  review  and  review 
of  patient  management  with  respect  to  diagnostic 
and  therapeutic  procedures? 

a.  To  what  extent  do  the  standards  and  criteria 
vary  among  PSROs? 

b.  Do  the  criteria  and  standards  adopted  by  PSROs 
and  institutions  screen  out  cases  for  further 
review? 

c.  What  are  the  key  factors  which  contribute  to 
variations  in  criteria  and  standards,  and  are 
such  variations  justified? 
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TABLE  2.  FORMATIVE  ISSUES 


(Continued) 


ISSUES 


PRIORITIES 


9.     Are  criteria  and  standards  being  developed  and 
reviewed  systematically  and  on  a  timely  basis 
and  are  revisions  implemented? 


a.     What  selection,  control,  and  review  processes 
are  utilized  to  ensure  the  timely  development 
of  new  and  revision  of  established  criteria 
and  standards? 


b.     What  factors  are  utilized  to  develop  new 
and  revise  established  criteria  and 
standards? 

10.    What  is  the  cost  for  development,  review,  and  B 
modification  of  criteria  and  standards? 


Notification,  review,  and  appeals 

11.  How  have  various  notification,  review  and  B 
appeals  processes  affected  the  health  care 

review  mechanisms? 

12.  Does  the  structure  of  the  notifications,  review,  B 
and  appeals  mechanism  allow  for  an  efficient 

and  judicious  system? 


Administration  and  organization 

13.    How  effectively  and  efficiently  are  the  various  A 
PSROs  organized  in  terms  of  their: 

a.  relationships  with  other  organizations 
(health  care  institutions,  state  councils,  sup- 
port centers,  etc.) , 

b.  Staffing  (number,  job  classifications, 
qualifications  of  personnel,  etc.), 

c.  delegation  of  functions  (e.g.,  review  authority] , 
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TABLE  2,  FORMATIVE  ISSUES   (Continued) 


i  hull  i  ur\imi  i  »  u.  ijjulj 

ISSUES 

PRIORITIES 

d.     review  activity  reimbursement  mechanisms? 

14. 

What  has  been  the  effectiveness  of  PSRO  self- 
pva  1  us  ti  on*,  7 

B 

15. 

Have  support  centers  effectively  provided  technical 
support  to  the  PSROs  during  their  planning,  condit- 
ional and  operational  stages? 

B 

16. 

Have  state  councils  been  effective  in  carrying 
out  their  functions? 

C 

17. 

Are  the  numbers  of  personnel  and  their  levels  of 
skill  and  training  appropriate  for  the  effective 
operation  of  the  various  program  elements? 

B 

18. 

How  efficient  and  effective  are  the  PSRO 
information  systems  with  respect  to  unit  costs, 
quality  of  the  data  base,  timeliness,  maintain- 
ability, and  flexibility? 

B 

19. 

How  does  the  effectiveness  and  efficiency  vary 
across  data  flow  models? 

B 

20. 

How  effective  is  the  data  confidentiality  system? 

B 
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TABLE  3 
SUMMATIVE  ISSUES 


ISSUES 


PRIORITIES 


1.  Are  all  Medicube  beneficiaries  covered  by 
activities  of  PSROs? 

a.  Have  PSRO  activities  been  implemented  in  all 
hospitals  which  participate  in  Medicube 
benefit  programs  and  which  are  located  in 
PSRO  areas  with  conditional  or  operational 

PSROs? 

b.  Are  all  Medicube  hospital  episodes  receiving 
PSRO  review? 

2.  Are  the  health  care  review  activities  (Ac,  CSR, 
CQA,  MCE  and  profiling),  as  carried  out  in  the 
PSRO  program,  assuring  the  quality  and 
appropriateness  of  the  utilization  of  health 
services? 

3.  What  has  been  the  impact  of  PSRO  activities  on 
physician  behavior? 


b. 


have  physicians1  patterns  of  practice 
changed  as  measured  by  deviations  from 
criteria  and  standards? 

Has  physician  participation  in  the  Medicube 
programs  changed  significantly? 

What  impact  have  organizational  and 
environmental  factors  had  on  changes  in 
physician  behavior? 


i 
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TABLE  3    SUMMATIVE  ISSUES 


(Continued) 


ISSUES 


What  has  been  the  impact  of  PSRO  activities  on 
institutional  behavior? 

a.  Has  the  organizational  structure  (e.g., 
committees  and  staffing  levels)  and  operat- 
ing procedures  (e.g.,  medical  records 
abstracting)  of  the  institutions  changed 
as  a  result  of  PSRO  activities? 

b.  Have  there  been  changes  in  the  physical 
structure  or  layouts  of  institutions  as  a 
result  of  PSRO  activities? 

c.  Has  there  been  a  significant  change  in 
participation  by  institutions  in  the  Medicube 
programs? 

d.  Do  hospital  occupancy  rates  indicate  that 
changes  in  Medicube  utilization  induced  by 
PSRO  review  have  been  offset  by  changes  in 
utilization  by  non-Medicube  populations? 

What  has  been  the  effect  of  PSRO  activities  on 
health  care  expenditures  in  the  United  States? 


a. 


What  have  been  the  changes 
expenditures  in  the  United 
respect  to: 


in  health  care 
States  with 


PRIORITIES 


b. 


(1)  total  health  care  expenditures, 

(2)  total  Medicube  expenditures, 

(3)  PSRO  program  expenditures? 

What  has  been  the  effect  of  PSRO  activities 
on  per  capita  Medicube  expenditures? 

What  have  been  the  changes  in  hospital 
expenditures  on  utilization  review  (UR) 
and  other  quality  assurance  activities  due 
to  the  introduction  of  PSROs? 
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TABLE  4 

SECONDARY  EFFECT  ISSUES 


ISSUES 

PRIORITIES 

'K 

What  has  been  the  impact  of  the  PSRO  program  on 

U  (rile  i    rcUci  a  1    JJ  i  Uy  raillb  ,   c  .  y  .  5    Our,   nHUo  »   lin  1  ,  ailU 

manpower? 

C 

0 

c  • 

Wild  l  nab  Uccii   uric   I  iiipau  t  u  i    trie  rjrvu  p ruy  rain  un 

health  regulation  and  health  regulatory  agencies? 

C 

3, 

What  has  been  the  effect  of  the  PSRO  program  on 
tho  diffusion  of  mpdiral  terhnol oav? 

B 

4. 

What  has  been  the  impact  of  PSRO  activities  on 
health  professional  education  and  physician 
participation  in  continuing  education? 

B 

5. 

What  has  been  the  effect  of  PSRO  activities  on 
health  care  institutions  with  regards  to  charges? 

B 

6. 

What  has  been  the  impact  of  PSRO  activities  on 
the  health  status  of  the  Medicube  and  non- 
Medicube  populations? 

A 
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TABLE  5 

CONDITIONS  USED  IN  THE  EVALUATION 
OF  THE  PSRO  PROGRAM 


ICDA-8 


Malignant  Neoplasm  Breast  174 

Diabetes  Mellitus  250 

Neuroses  300 

Cataract  374 

Hypertension  400  -  404 

Acute  Myocardial  Infarction  410 

Chronic  Ischemic  Heart  Disease  412 

Congestive  Heart  Failure  427.0,  427.1 

Cerebrovascular  Disease  430  -  438* 

Pneumonia  480  -  486* 

Hypertrophy  Tonsils  &  Adenoids  500 

Ulcer  531  -  534 

Acute  Appendicitis  540  -  542 

Inguinal  Hernia  without  obstruction  550 

Cholelithiasis/cystitis  574,  575 

End  Stage  Renal  Disease  580,  582,  753.1, 

759.8,  590.0,  590.1, 
446.2,  752,  753 

Urinary  Tract  Infection  590,  595 

Hyperplasia  Prostate  600 

Delivery  with  and  without  complications  650  -  662 

Intervertebral  Disc  Displacement  725 
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TABLE  5  -  cont. 

ICDA-8 

Fracture  Femur 

Complications  and  Adverse  Reactions 
Hysterectomy 


820 

960  -  979 
995  -  999 

69.1  -  69.7 


*Includes  unspecified  cases 


Chapter  IV 
Specification  of  Evaluation  Issues 
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CHAPTER  IV 
SPECIFICATION  OF  EVALUATION  ISSUES 


The  two  preceding  chapters  have  specified  (1)  the  issues  that  were 
considered  and  selected  for  the  evaluation  of  the  PSRO  program  and 
(2)  the  questions  whose  answers  define  the  detailed  issue  specifications 
This  chapter  contains  the  detailed  specifications  for  each  of  the  13 
selected  (A  priority)  evaluation  issues.    The  issues  are  presented 
according  to  issue  classification  (interim,  formative,  summative,  and 
secondary  effects)  and  are  as  follows: 


Interim  Issue 
1 

Pages  23  -  26 


Have  the  PSRO  program  elements  (PSROs,  support 
centers,  State  Councils)  been  developed  and 
placed  into  operation  as  planned? 


Formative  Issue 
1 

Pages  27  -  46 


Have  admission  certification  and  continued  stay 
review  (concurrent  review)  been  effective  methods 
of  reducing  medically  unnecessary  hospital 
utilization? 


Pages  47  -  52 


Is  concurrent  quality  assessment  (CQA),  as  carried 
out  1n  the  PSRO  program,  an  effective  method  of 
assuring  the  appropriate  utilization  of  services 
(diagnostic  and  therapeutic  procedures)? 


Pages  53  -  63 


Is  retrospective  health  care  review,  as  carried 
out  in  the  PSRO  program,  an  effective  method  of 
assessing  the  quality  and  appropriateness  of  the 
utilization  of  services? 


Pages  64  -  70 


How  effective  are  the  various  corrective  mechanisms 
applied  to  institutions  or  practitioners  for 
correcting  inappropriate  utilization  of  services 
and/or  improving  the  quality  of  care? 
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9  Are  criteria  and  standards  being  developed  and 

p        71  _  75        reviewed  systematically  and  on  a  timely  basis 
9  and  are  revisions  implemented? 


13  How  effectively  and  efficiently  are  the  various 

Pages  76  -  79        PSROs  organized? 

Summative  Issues 


1  Are  all  Medicube  beneficiaries  covered  by 
Pages  80  -  82        activities  of  PSROs? 

2  Are  the  health  care  review  activities  (AC,  CSR, 
CQA,  MCE  and  profiling),  as  carried  out  in  the 

Page  83  PSRO  program,  assuring  the  quality  and  appropriateness 

of  the  utilization  of  health  services? 


3  What  has  been  the  impact  of  PSRO  activities  on 

Pages  84  -  89        physician  behavior? 


4  What  has  been  the  impact  of  PSRO  activities  on 

Pages  90  -  95        institutional  behavior? 


5  What  has  been  the  effect  of  PSRO  activities  on 

Pages  96  -  101       health  care  expenditures  in  the  United  States? 


Secondary  Effects 
Issue 


6 

Page  102 


What  has  been  the  impact  of  PSRO  activities  on  the 
health  status  of  the  Medicube  and  non-Medicube 
populations? 
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INTERIM  ISSUES 


Issue  #1:  HAVE  THE  PSRO  PROGRAM  ELEMENTS  (PSROS,  SUPPORT  CENTERS,  AND 
STATE  COUNCILS)  BEEN  DEVELOPED  AND  PLACED  INTO  OPERATION  AS 
PLANNED? 


A.  Sub-Issues 

1.  What  is  the  status  of  the  program  in  terms  of  coverage  (geo- 
graphic and  beneficiary  population),  basic  implementation 
schedule,  and  cost? 

2.  How  far  advanced  is  the  PSRO  program  in  terms  of  the  progress 
that  has  been  made  toward  implementing  the  PSRO  functions 
(e.g.,  AC,  CSR,  MCE,  Profiles,  etc.)? 

Rationale: 

It  is  impossible  to  measure  fully  the  effectiveness  of  the  PSRO  program 
in  the  short  run.    Effectiveness  of  the  program  development  effort  is 
contingent  upon  (a)  putting  in  place  appropriate  program  elements  and 
(b)  impacting  on  program  goals  at  agreed  upon  levels  of  effectiveness. 
This  latter  aspect  will  not  be  measurable  until  the  program  has  had 
sufficient  time  to  mature. 

Certain  process  issues,  however,  may  be  evaluated.    At  the  simplest 
level,  it  may  be  determined  if  program  elements  have  been  developed  and 
placed  into  operation  on  schedule.    Barriers  to  development  may  be 
isolated,  and  strategies  to  overcome  the  barriers  developed. 

B.  Management/ Pol  icy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Recommend  changes  in  the  program  development  schedule,  both  with 
respect  to  time  and  numbers  of  program  elements. 

2.  Monitor  individual  PSRO  function  development  more  closely  for 
consideration  of  contract  continuation. 

3.  Provide  additional  technical  assistance  to  a  particular  program 
element  or  groups  of  such  elements. 

C.  Analysis 


Analysis  will  be  based  on  a  review  of  contract  applications,  contracts,  PSRO 
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information  system  (PMIS)  reports  and  progress  reports  from  the  PSROs, 
State  Councils  and  support  centers.    Progress  of  each  PSRO  will  be 
compared  to  its  implementation  plan.    The  following  measures  of  the 
overall  program's  development  will  be  obtained  and  compared  to  the 
expected  program  progress  schedule:    (Table  6,  Items  1-5) 


(a)  Number,  percent  of  total,  geographic  distribution  and 
operating  status  (e.g.,  planning,  conditional,  operational) 
of  PSROs,  State  Councils,  and  support  centers. 

(b)  Size  and  percent  of  total  population  covered  by  program. 

(c)  Number  and  percent  of  total  hospital  admissions  reviewed. 

(d)  Budget  of  each  PSRO, 

(e)  Number  and  category  of  diagnoses  for  which  standards  and 
criteria  have  been  developed. 


In  addition,  the  status  of  the  following  designated  PSRO  functions  will 
be  measured  in  terms  of  their  number,  amount  of  time  they  have  been  fully 
operational,  and  costs  (Table  6,  Item  6): 

(a)  admission  certification 

(b)  continued  stay  review 

(c)  medical  care  evaluations  studies 

(d)  practitioner,  institutional,  and  patient  profiles 

(e)  external  relations 

(f)  internal  relations 

(g)  standards  and  criteria  development 

(h)  reviews  and  appeals 

(i)  management  (e.g.,  staffing) 
(j)  delegation  of  review 

(k)  data  systems 

(1)  continuing  education 

(m)  LTC  review 

(n)  ambulatory  review 


Those  PSROs  who  are  considerably  behind  in  meeting  their  scheduled 
milestones  will  be  surveyed  to  determine  the  reasons  for  the  delay. 
In  this  way  the  type  and  level  of  technical  assistance  required  to 
correct  the  problems  can  be  assessed. 


i 
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FORMATIVE  ISSUES 

Issue  #1:  HAVE  ADMISSION  CERTIFICATION  AND  CONTINUED  STAY  REVIEW 
(CONCURRENT  REVIEW)  BEEN  EFFECTIVE  METHODS  OF  REDUCING 
MEDICALLY  UNNECESSARY  HOSPITAL  UTILIZATION? 

A.  Sub-Issues 

1.  To  what  extent  has  AC  resulted  in  reducing  medically  unnecessary 
admissions? 

2.  To  what  extent  has  CSR  resulted  in  reducing  excessive  lengths 
of  stay? 

3.  How  does  the  effectiveness  of  concurrent  review  vary  with  respect  to 

(a)  PSROs, 

(b)  Institutions, 

(c)  AC  models  (e.g.,  preadmission  certification) 

(d)  CSR  models  (e-g«>  alternative  percentile  cutoffs  for  continued 
stay  review)? 

4.  To  what  extent  has  CSR  resulted  in  medically  inappropriate  denials 
of  extension? 

5.  What  has  been  the  impact  of  PSRO-induced  changes  in  hospital 
utilization  in  terms  of  Med i cube  expenditures  for  health  care? 

6.  Has  utilization  control  through  concurrent  review  resulted  in 
net  savings? 

Rationale: 

It  is  currently  believed  that  a  significant  portion  of  Medicube  hospital 
utilization  is  not  medically  necessary.    If  concurrent  review  is  effective 
it  should  result  in  a  decrease  in  Medicube  hospital  utilization  through 
reduction  of  this  unnecessary  component.    It  is  therefore  important  to 
measure  this  effect  and  to  test  for  its  uniformity  across  PSROs  and  insti- 
tutions.   Because  varying  approaches  will  be  used  to  implement  concurrent 
review  (e.g.,  preadmission  certification),  it  will  also  be  important  to 
identify  the  approaches  which  particularly  enhance  (or  reduce)  the  effective 
ness  of  this  component  of  the  PSRO  program. 
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Utilization  control  through  concurrent  review,  however,  may  create 
problems.    Premature  discharge  may  reflect  itself  in  longer  recovery 
times,  in  inappropriate  variations  in  treatment,  or  in  excessive 
ambulatory  care.    It  also  might  be  reflected  in  hospital  readmissions. 
If  there  is  a  tendency  to  discharge  patients  too  early,  this  practice 
might  be  indicated  by  higher  rates  of  readmission  for  the  same  or  related 
conditions  within  a  short  time  after  discharge.    Thus,  if  an  inappropriately 
short  length  of  stay  criterion  is  enforced,  the  readmission  rates  might 
show  an  Increase.    It  is  therefore  necessary  to  monitor  readmission  rates 
to  determine  if  short  stays  are  producing  untoward  results.    (The  monitoring 
of  individuals'  receipt  of  health  services-- inpatient,  ambulatory  and  long 
term  —  necessary  to  conduct  an  examination  of  possible  similar  effects  of 
the  AC  component  of  the  program  would  be  extremely  costly  and  would  involve 
the  risk  of  considerable  infringement  of  personal  privacy.    As  a  result, 
such  an  examination  will  not  be  conducted  as  part  of  this  evaluation.) 

Finally,  measurement  of  the  effectiveness  of  concurrent  review  must  be 
analyzed  related  to  the  costs  incurred.    It  is  therefore  necessary  to 
estimate  in  dollar  terms  the  impact  of  utilization  changes  induced  by  PSRO 
review,  and  to  compare  these  "benefits"  to  the  expenditures  for  concurrent 
review. 

B.  Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Change  admission  criteria. 

2.  Change  LOS  and/or  extension  certification  criteria  and  standards 
for  specific  diagnoses. 

3.  Recommend  to  the  PSROs  the  adoption  of  a  specific  AC  model  or  models. 

4.  Recommend  to  the  PSROs  the  adoption  of  a  specific  CSR  model    or  models. 

5.  Monitor  certain  institutions  or  PSROs  more  closely. 

C.  Analysis 

While  methodological  and  data  problems  are  not  uncommon  to  the  analyses 
presented  in  this  Plan,  they  are  particularly  acute  with  respect  to  the 
analyses  of  this  issue.    Two  of  these  problems  were  thought  to  be  sufficiently 
important  to  warrant  special  consideration  as  a  preface  to  the  analyses. 

A  significant  data  problem  is  caused  by  the  following  situation:  (a)  there 
is  a  wide  variety  of  health  services  which  may  be  provided  during  an  episode 
of  care,  however,  the  setting  of  the  delivery  of  each  service  is  not  always 
and  not  often  dependent  on  the  service;  (b)  the  recording  of  the  provision 
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of  services  is  fragmented  according  to  setting,  and  the  quality  of  the 
data  varies  by  setting;  (c]  there  exists  no  national  system  for  the 
collection  of  comprehensive,  integrated  records  of  Medicube  utilization 
regardless  of  data  quality;  and  (d)  because  of  the  complexity  of  the  PSRO 
program  and  its  limited  initial  authority,  reviews  and  data  collection 
will  focus  only  on  care  provided  in  hospitals. 

Thus  while  the  PSRO  program  is  expected  to  have  spillover  impacts  on 
ambulatory  and  long  term  care,  adequate  data  to  support  measurement  and 
evaluation  of  these  impacts  at  a  national  level  will  not  be  available. 
Collection  of  such  data  -  operational  and  particularly  baseline  -  is  be- 
yond the  scope  of  the  PSRO  program,  much  less  the  PSRO  Evaluation.  (Some 
comprehensive  data  is  available  on  Medicaid  patients  in  those  states  which 
have  implemented  the  Medicaid  Management  Information  System,  MMIS.  Few 
states,  however,  have  had  long  experience  with  the  MMIS,  and  those  states 
which  have  had  the  experience  are  not  typical  of  the  states  as 
a  whole.    The  self-selection  bias  inherent  in  the  use  of  such  data  in  a 
national  program  evaluation  might  seriously  compromise  the  results  of  that 
evaluation,  and  as  a  result,  the  use  of  this  data  has  not  been  specified  in 
this  Evaluation  Plan.    However,  so  that  the  possible  insights  available 
through  the  examination  of  such  data  will  not  be  forgone,  further  con- 
sideration will  be  given  to  special  studies,  where  possible,  on  observed 
changes  in  patient  profiles  available  through  the  MMIS). 

Because  of  the  lack  of  comprehensive,  national  data,  the  analyses  are 
focused  on  measures  of  inpatient  hospitalization.    However,  although  at 
the  present  time  the  spillover  impacts  may  be  immeasurable,  they  have  been 
given  consideration,  where  possible,  in  the  design  of  the  analyses. 

A.  second  major  problem  particular  to  the  analysis  of  this  issue  involves 
the  definition  and  measurement  of  "unnecessary"  care.    It  has  been  suggested 
that  a  direct  approach  be  used  to  determine  the  extent  to  which  concurrent 
review  reduces  medically  unnecessary  utilization.    While  such  methods  have 
been  successfully  designed  for  use  in  individual  institutions ,1/  the  use  of 
such  a  methodology  in  a  national  evaluation  program  poses  a  number  of  pro- 
blems.   Developing  a  widely  acceptable  algorithm  is  the  prime  example. 
Conceptions  for  identifying  unnecessary  care  are  both  numerous  and  diverse. 2/ 
No  one  of  thse  concepts  seems  so  obviously  better  than  all  others.  However, 
even  if  there  were  such  a  methodology,  its  use  would  be  questionable  in  the 
PSRO  program  because  it  implies  the  development  of  national  standards,  a  con- 
cept contrary  to  the  PSRO  legislation. 

1/  Cf.  Zimmer,  "Length  of  stay  and  hospital  bed  misutil ization," 

"Medical  Care",  May  1974.    Also  see  the  works  of  Brook,  Gertman,  and  others. 

2/  Cf.  Boulding,  "The  concept  of  need  for  health  services,"  Milbank  Memorial 
Fund  Quarterly,  October  1966. 
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Another  methodological  problem  is  the  use  of  concurrent  review  denials 
as  a  surrogate  measure  of  unnecessary  care  as  has  been  suggested  by 
several  sources.    While  such  denials  will  be  meaningful  as  instances  of 
proposed  care  judged  not  medically  necessary,  these  denials  cannot  be 
aggregated  to  form  an  adequate  measure  of  unnecessary  care  that  would  have 
been  provided  in  the  absence  of  the  PSRO  program.    The  major  reason  for  this 
is  that  the  denials  themselves  have  both  educational  and  deterrent  effects. 
These  effects  tend  to  inhibit  the  provision  of  unnecessary  care,  however, 
neither  effect  is  directly  measurable.    Thus,  the  number  of  denials  would 
underestimate  unnecessary  care  averted  as  a  result  of  PSRO  activities. 
Finally,  the  absence  of  baseline  denial  statistics  further  lessens  the 
usefulness  of  denials  for  the  examination  of  unnecessary  care.  (Denials 
are  meaningful,  however,  in  the  examination  of  changes  in  physician  behavior. 
See  Formative  Issue  #4.) 

As  a  result  of  these  factors  the  approaches  to  be  used  in  the  evaluation 
of  concurrent  review  are  indirect  ones. 

1.    Description  of  the  Analyses 

Ideally,  for  the  analysis  of  this  issue,  one  would  conduct  a  well  controlled 
experiement  where  the  observed  changes  in  utilization  would  be  due  solely 
to  the  presence  or  absence  of  PSRO  activities.    Such  a  level  of  control  would 
be  difficult,  if  not  impossible,  to  achieve.    It  may  be  possible,  however, 
to  control  for  enough  critical  factors  to  allow  for  at  least  the  conduct  of 
a  meaningful  quasi-experiment,  The  analyses  which  follow  have  been  designed 
such  that  they  can  be  put  into  the  context  of  a  quasi-controlled  study,  where 
possible.    The  results  of  the  analyses,  however,  are  not  totally  dependent 
upon  the  ability  to  conduct  a  quasi-controlled  study.    Should  the  data  in- 
dicate that  the  study  is  insufficiently  controlled,  the  analysis  will  still 
be  capable  of  providing  measures  of  the  impact  of  PSRO  on  utilization. 

The  analyses  of  the  first  sub-issue  and  parts  of  three  other  sub-issues  will 
be  conducted  in  the  course  of  a  quasi-controlled  experiment  involving  a  sample 
of  PSRO  areas  which  have  a  conditional  PSRO  (designated  in  calender  year 
1975)  and  a  matched  sample  Vof  PSRO  areas  which  have  not  had  a  PSRO  (in  any 
stage)  designated  before  the  end  of  1976.    (The  matching  will  be  based  on 
characteristics  of  the  PSRO  areas  such  as  number  of  Medicube  admissions,  number 
of  hospitals,  population  density,  etc.)    These  analyses  will  be  based  on 
comparisons  of  utilization  changes  which  are  observed  in  calender  years  1976 
and  1977.    The  sources  of  data  will  be  the  PSRO  Discharge  Data  Set  (PHDDS) 
in  the  active  PSRO  areas,  and  collected  Uniform  Hospital  Data  Abstract  (UHDA) 
records  and  supplementary  data  for  Medicare  and  Medicaid  in  the  non-active 
areas. 

3/  Hereafter,  in  descriptions  of  analyses  involving  comparisons  of  matched 
PSRO  areas,  the  term  "active    PSRO  area  "  will  be  used  to  denote  a  PSRO 
area  which  has  a  conditional  or  operational  PSRO.    The  term  "non-active 
PSRO  area"  will  be  used  to  denote  a  PSRO  area  which  has  not  had  designated 
an  operational,  conditional,  or  planning  PSRO. 
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(PSROs  in  states  which  initiate  changes  in  Medicaid  benefit  packages 

or  eligibility  rules  will  be  excluded  from  the  analysis  unless  they  can  be 

matched  with  PSROs  in  the  same  state.) 

Reduction  of  Medically  Unnecessary  Admissions 

This  analysis  will  be  based  on  longitudinal  studies  of  diagnostic  specific 
and  population  based  admission  rates.    If  some  percentage  of  acute 
hospitalizations  are  not  medically  necessary,  the  institution  of  an  effective 
AC  program  should  at  least  initially  reduce  admission  rates  through 
elimination  of  the  "unnecessary"  component.    Therefore,  relative  changes  in 
admission  rates  between  time  periods  will  be  the  primary  impact  measure. 

Baseline  data  in  each  PSRO  area  should  provide  information  on  diagnostic- 
specific  and  population  based  admission  rates  per  quarter  for  at  least  one 
year's  time.    (Admission  rates  per  quarter  are  necessary  to  enable  adjust- 
ments for  seasonal  variations.)    This  will  require  that  the  baseline  data 
include  estimates  of  the  size  of  the  corresponding  Medicube  populations. 
In  the  choice  of  diagnoses  to  be  used  in  this  analysis  preference  is  given 
to  those  diagnoses  which  have  demonstrated  relatively  stable  admission  rates 
in  the  recent  past.    This  is  done  to  eliminate  as  much  random  fluctuation 
as  is  possible  for  the  analysis  of  changes  in  the  admission  rates.  Changes 
in  admission  rates  will  be  computed  as  the  difference  between  observed  rates 
in  1976  and  1977.    It  should  be  noted  that  these  baseline  data  may  be  difficult 
to  obtain,  especially  those  dealing  with  the  non-Medicare  population. 

All  diagnoses  selected  for  the  study  will  be  combined  to  provide  a  measure 
of  the  change  in  the  rate  of  hospitalization.    That  measure  will  be  a  weighted 
average  of  the  percentage  changes  in  admission  rates.    The  weights  to  be 
used  will  be  based  on  the  relative  frequencies  or  incidences  of  those 
diagnoses  under  study  in  relation  to  all  admitting  diagnoses. 

The  difference  in  the  estimated  percentage  changes  in  admission  rates  between 
the  PSRO  active  areas  and  the  non-active  areas  will  be  the  primary  evaluation 
measure  of  the  impact  of  admission  certification  under  PSRO.    (Table  7,  Item  1) 
Further,  the  total  number  of  Medicube  days  of  care  multiplied  by  the  ratio  of 
this  difference  in  percentage  changes  to  the  value  one  minus  this  difference 
will  be  an  estimate  of  the  unnecessary  days  of  care  averted  through  the  AC 
process.    (Table  7,  Item  3) 

Changes  in  Lengths  of  Stay 

This  analysis  will  be  based  on  longitudinal  studies  of  diagnostic  or  pro- 
cedure specific  average  lengths  of  stay  for  cases  certified  for  admission. 
Relative  changes  in  average  lengths  of  stay  will  be  the  primary  impact 
measure  because  if  some  fraction  of  the  days  of  care  in  Medicube  hospitali- 
zations are  not  medically  necessary,  an  effective  CSR  program  should  at  least 
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initially  reduce  the  average  lengths  of  stay  through  a  reduction  in  the 
"unnecessary"  component.    (Other  things  being  equal,  an  effective  AC  pro- 
gram should  tend  to  lengthen  the  average  length  of  stay  by  elimination  of 
unnecessary^  and  most  likely  short,  stays.    Excessive  lengths  of  stay, 
however,  are  not  limited  to  cases  with  long  lengths  of  stay  so  that  in 
the  absence  of  an  extraordinarily  effective  AC  program  the  net  effect  of 
concurrent  review  should  in  fact  be  to  shorten  average  lengths  of  stay.) 

The  analysis  will  be  based  on  changes  in  diagnostic  or  procedure  specific 
average  lengths  of  stay  between  1976  and  1977.    All  diagnoses  selected  for 
the  study  will  be  combined  to  provide  a  measure  of  the  change  in  the  lengths 
of  stay.    This  measure  will  be  a  weighted  average  of  the  percentage  changes 
in  lengths  of  stay.    (Table  7,  Item  2)    The  weights  to  be  used  will  be  based 
on  the  relative  total  days  of  care  for  all  Medicube  beneficiaries  with  respect 
to  the  individual  diagnoses  or  procedures.    (Discharges  due  to  deaths  will 
be  excluded  from  the  calculations  of  average  lengths  of  stay.) 

The  difference  in  the  estimated  percentage  changes  in  average  lengths  of 
stay  between  the  PSRO  active  areas  and  the  non-active  areas  will  be  the 
primary  evaluation  measure  of  the  utilization  impact  of  continued  stay  review 
under  PSRO.    The  total  number  of  Medicube  days  of  care  multiplied  by  the 
ratio  of  this  difference  in  percentage  changes  to  the  value  one  minus  this 
difference  will  be  an  estimate  of  the  number  of  unnecessary  days  of  care 
averted  through  the  CSR  process.    (Table  7,  item  4) 

In  order  to  examine  the  uniformity  of  CSR  impact  two  additional  analyses 
will  be  performed  using  the  data  collected  for  this  study.    In  the  first, 
the  individual  diagnoses  selected  for  the  study  will  be  grouped  according 
to  the  following  categories: 

(1)  surgical 

(2)  medical-surgical 

(3)  medical 

(4)  mental 

Estimates  of  the  relative  changes  in  lengths  of  stay  (Table  7,  Item  2)  will 
be  computed  for  each  of  these  more  general  diagnostic  classes.    The  second 
analysis  will  involve  grouping  the  observations  according  to  whether  or 
not  multiple  diagnoses  were  recorded.    Estimates  of  the  relative  changes  in 
lengths  of  stay  will  be  computed  for  both  groups.    Significant  differences 
between  the  values  of  these  additional  measures  may  indicate    areas  where 
CSR  effectiveness  can  be  improved. 
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Additional  Quasi-Experimental  Analyses 

The  results  of  the  AC  and  CSR  analyses  will  be  checked  for  accuracy  by 
comparing  the  change  in  utilization  rates  (expressed  in  terms  of  Medicube 
days  of  care  per  eligible)  in  the  PSRO  active  areas  to  the  change  observed 
in  the  non-active  areas.    The  difference  between  these  two  measures 
multiplied  by  the  number  of  Medicube  eligibles  in  the  active  areas  in  the 
second  year  will  yield  an  estimate  of  days  of  care  averted  through  the  PSRO 
concurrent  review  process.    This  estimate  should  be  consistent  with  the 
sum  of  the  estimates  from  the  AC  and  CSR  analyses.    (Table  7,  Item  5) 

The  impact  of  these  changes  in  utilization  on  Medicube  expenditures  will 
be  estimated  as  the  product  of  the  average  daily  room  and  board  charge  for 
the  PSRO  area  and  the  estimate  of  reduction  of  Medicube  days  of  care. 
(Table  7,  Item  6)    This  approach  should  give  a  more  accurage  estimate  of 
savings  than  one  based  upon  observed  average  costs  per  day  since  fewer 
services  in  addition  to  room  and  board  are  provided  toward  the  end  of  hospital 
stays  than  are  provided  at  the  beginning.    The  estimate  of  AC  savings  will 
be  biased  downwards,  but  the  next  most  plausible  alternative  approach  (based 
on  changes  in  admission  rates  and  diagnostic  specific  imputed  expenditures 
per  admission)  would  be  heavily  biased  upwards.    The  lower  estimates  are 
preferable  because  ambulatory  care  or  long  term  care  will  be  substituted 
for  inpatient  utilization  averted  through  the  AC  and  CSR  processes,  thus 
only  part  of  the  estimated  savings  will  be  realized.    To  the  extent  that 
changes  in  utilization  induce  price  increases  greater  than  those  that  would 
have  occurred  in  the  absence  of  concurrent  review,  these  estimates  will 
overstate  the  magnitude  of  savings.    However,  it  is  not  possible  at  this 
time  to  estimate  the  probability  of  such  an  effect  nor  its  magnitude. 

An  analysis  and  comparison  of  benefit  cost  ratios  of  concurrent  review  under 
PSRO  and  concurrent  review  under  the  utilization  review  regulations  would 
be  of  great  use  in  an  examination  of  public  policy  in  this  area.  Unfortunately 
such  an  analysis  cannot  be  conducted  in  the  context  of  the  quasi -control led 
study  because  (1)  many  methodological  compromises  would  have  to  be  made  for 
the  analysis,  in  particular,  comparing  the  "products"  of  utilization  review 
and  PSRO  given  the  additional  focus  on  quality  in  the  latter,  and  (2) 
difficulties  in  acquiring  adequate  data  in  non-active  PSRO  areas.  (An 
analysis  of  variations  in  estimated  benefit-cost  ratios  for  active  PSROs  is 
described  in  the  analysis  of  Net  Savings.) 

One  final  analysis  will  examine  the  impact  of  characteristics  of  the  PSROs 
on  the  effectiveness  of  concurrent  review.    The  ratio  of  days  of  care 
averted  to  Medicube  days  of  care  provided  will  be  used  as  a  dependent  variable 
in  a  regression  analysis.  4/    Policy  variables  included  in  the  analysis  are 
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4/    In  the  analysis  of  several  of  the  issues  in  the  Plan  it  is  important 
to  measure  the  individual  impacts  of  several  factors  which  jointly 
affect  an  indicator  of  proqram  performance.    In  such  instances 

regression  analysis  has  been  specified.    This  involves  the  use  of 
statistical  techniques  which  will  identify  the  direction  and  magnitude 
of  the  various  impacts  to  be  examined,  as  well  as  the  validity  of  the 
relationships  under  consideration.     The  relationships  implied  in 
the  analyses  are  linear  ones,  however,  where  the  data  suggest  a 
specific  non-linear  relationship,  this  can  be  substituted  in  the 
analyses.    Similarly,  where  the  data  suggest  the  use  of  alternative 
multivariate  techniques,  such  techniques  can  be  used  to 
supplement  and  to  test  the  analyses  currently  specified  in 
the  Plan. 

Because  of  the  nature  of  the  data  to  be  used  two  particular  techniques 
of  regression  analysis  must  be  employed.    The  first  involves  the  use 
of  instrumental  variables  or  two-stage  least  squares  regressions. 
This  will  correct  for  intercorrelations  among  the  variables  under 
consideration  which  otherwise  would  reduce  the  precision  of  the  results. 
The  second  technique  involves  weighting  of  the  observations  (usually 
by  a  measure  of  caseload).    This  will  correct  for  differences  in  the 
accuracy  of  the  estimated  variances  of  the  observations  which  otherwise 
would  also  tend  to  reduce  the  precision  of  the  results. 

Three  classes  of  independent  variables  are  identified  in  the  description 
of  each  regression:    policy  variables,  "qualifying"  variables,  and 
"literature"  variables.    Policy  variables  represent  aspects  of  the 
PSRO  system  which  are  manipulable  through  policy  decisions,  e.g., 
delegated  review  authority,  and  unit  review  expenditures.  "Qualifying" 
variables  represent  aspects  of  the  PSRO  system  which  cannot  be  manipulated 
through  policy  decisions  but  which  must  be  taken  into  account  when 
assessing  PSRO  experience.    Examples  are  the  number  of  hospitals 
in  a  particular  PSRO  area,  and  whether  the  PSRO  area  is  in  a  single 
or  multiple  PSRO  state.    Certain  other  variables  included  in  the 
analysis  are  designated  as  "literature"  variables  because  they  have 
been  demonstrated  in  health  services  literature  to  be  important  in 
relationships  of  the  types  specified  in  the  Evaluation  Plan.  Examples 
in  this  category  are  population  density,  and  the  number  of  hospital 
beds.    Finally,  certain  variables  which  appear  in  the  literature  have 
been  excluded.    The  major  considerations  in  these  exclusions  were  the 
additional  cost  of  data  collection  and  processing  of  the  variables 
and  the  fact  that  some  of  these  variables  could  not  be  well  specified 
or  were  believed  to  be  still  in  the  realm  of  research.    Examples  are 
sociodemographic  variables  with  respect  to  changes  in  utilization  and 
physician  specialty  and  training  variables  with  respect  to  concurrent 
review  denials. 
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the  percentage  of  Medicube  discharges  treated  in  delegated  hospitals  and 
concurrent  review  expenditures  per  Medicube  eligible.    Qualifying  variables 
include  the  number  of  hospitals  in  the  PSRO  area,  the  annual  number  of 
Medicube  discharges,  the  percentage  of  Medicube  discharges  treated  in 
hospitals  affiliated  with  medical  schools,  and  the  major  census  regions. 
Literature  variables  include  the  area-wide  occupancy  rate,  population 
density,  and  the  ratio  of  the  number  of  extended  care  beds  to  the  population 
older  than  65.    The  regressions  will  use  weights  based  on  the  Medicube 
days  of  care  in  each  area,  (Table  7,  Item  7) 

Variations  Across  Institutions  and  Concurrent  Review  Models 

Variation  in  approaches  to  concurrent  review  (e.g.,  preadmission 
certification  or   aternative  percentile  cutoffs  for  CSR)  will  be  more 
evident  within  PSRO  areas  than  between  PSRO  areas.    Institutional  com- 
parisons will  be  necessary  to  identify  which  of  these  approaches  are  the 
most  effective.    While  this  situation  will  prevent  comparisons  based  on 
population-based  utilization  rates  it  will  facilitate  an  analysis  which 
will  also  identify  the  impacts  of  institutional  and  environmental  factors 
on  the  effectiveness  of  concurrent  review. 

This  analysis  will  be  performed  twice,  (at  the  completion  of  calendar 
years  1976  and  1978)  and  will  be  based  on  observations  from  individual 
institutions.    Data  for  the  analysis  will  be  derived  from  PHDDS  records 
and  from  intermediary  and  carrier  records  as  necessary.    Initially,  all 
hospitals  in  active  PSRO  areas  will  be  included. 

The  focus  of  the  analysis  will  be  on  changes  in  Medicube  days  of  care  in 
each  institution.    The  percentage  change  between  each  hospital's  two  most 
recent  annual  total  of  Medicube  days  of  care  will  be  used  as  the  dependent 
variable  in  a  regression    analysis.    Policy  variables  included  in  the  analysis 
are  concurrent  review  expenditures  per  Medicube  discharge  and  dummy  variables 
to  represent  delegated  review  authority  and  the  various  concurrent  review 
model  types.    Qualifying  variables  include  the  PSRO  cohort,  5/  the  per- 
centage change  in  Medicube  eligibles,  population  density  and  the  ratio  of 
the  number  of  extended  care  beds  to  the  population  older  than  sixtyfive. 
Literature  variables  include  bed  size,  percentage  change  in  bed  size,  owner- 
ship, the  occupancy  rate,  the  percentage  change  in  average  daily  charges, 
and  affiliation  with  a  medical  school.    (Table  7,  Item  8) 


5/  PSRO  cohorts  will  be  defined  by  the  stages  of  national  PSRO  program 
implementation.    The  original  group  of  13  conditional  PSROs  form  the 
first  cohort.    Regression  analyses  which  include  the  cohorts  as  dummy 
variables  may  identify  differences  between  these  groups  of  PSROs. 
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Extension  Denials  and  Readmissions 

The  analysis  of  extension  denials  and  readmission  will  be  conducted 
as  a  special  study  to  be  initiated  early  in  calender  year  1977.  The 
study  will  be  conducted  in  a  sample  of  PSRO  areas  stratified  by  region. 
Information  needed  for  the  analysis  will  be  identified  through  examination 
of  local  PSRO  records  of  denials  and  subsequent  examination  of  PHDDS  re- 
ords  maintained  at  the  local  PSRO  level.    No  individual  identifiers  will 
be  reported  in  the  analysis  and  all  necessary  steps  will  be  taken  to 
assure  confidentiality. 

The  analysis  will  be  based  on  readmission  rates  for  the  same  or  a  related 
condition  or  complications.    Readmission  data  will  be  gathered  based  on 
the  90  days  following  discharge  and  will  be  cross  tabulated  by  (a)  whether 
the  length  of  stay  for  the  first  admission  was  under,  at,  or  over  the  norm 
for  that  diagnostic  category  (and  age  and  sex  where  necessary)  and  (b) 
whether  or  not  an  extension  denial  occurred  in  the  case.  Statistically 
significant  changes  in  the  proportions  of  these  two  categories  will  be 
monitored  as  will  be  the  changes  in  the  proportion  of  both  categories  to 
the  total  cases  within  each  condition.    If  the  proportion  of  cases  being 
readmitted  rises,  it  could  be  construed  as  an  indication  of  untoward  effects 
resulting  from  CSR,  while  its  remaining  the  same  could  indicate  the  absence 
of  such  effects.    Its  decrease  could  be  taken  to  be  an  indication  of  improved 
quality  of  care.    If  the  proportion  of  cases  being  readmitted  within  the 
sub-group  which  were  denied  extensions  rises,  it  could  be  construed  as  an 
indication  of  the  inappropriateness  of  extension  denial  activities.  A 
decrease  in  this  proportion  could  be  construed  as  an  indication  that  ex^ 
tension  certification  is  having  its  desired  effect.    The  absence  of  any 
change  could  be  interpreted  as  the  absence  of  such  effects. 

As  a  further  analysis  of  extension  denials,  frequency  distributions  of 
denials  will  be  reported  for  the  following  discharge  categories: 

(1)  Patient  remained  in  institution,  denial  overturned  on  reconsideration 
or  appeal ; 

(2)  patient  remained  in  institution,  not  eligible  for  reimbursement; 

(3)  patient  discharged  or  transferred  to  lower  level  of  care; 

(4)  patient  discharged  to  home.    (Table  7,  Item  10) 

Impact  on  Expenditures 

The  expenditure  analysis  described  in  the  quasi-experiment  section  was 
based  on  the  expenditure  impact  of  PSRO  relative  to  that  of  utilization 
review.    An  analysis  of  the  absolute  level  of  impact  will  also  be  of  use, 
and  such  an  analysis  can  be  performed  using  data  from  all  PSROs  rather 
than  just  the  PSROs  in  the  quasi-experiment. 
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This  analysis  will  be  performed  twice  Cat  the  completion  of  calendar 
years  1976  and  1978)  and  will  be  based  on  observations  obtained  from 
individual  PSROs.    Data  for  the  analysis  will  be  derived  from  PHDDS 
records,  from  Medicube  eligible  files,  and  from  intermediary  and  carrier 
records  as  necessary.    All  active  PSRO  areas  will  be  included  in  the  analysis. 

For  every  PSRO  area  an  annual  utilization  rate  will  be  computed  for  each 
of  the  two  twelve  month  periods  immediately  prior  to  the  study.  These 
rates    will  be  expressed  in  terms  of  Medicube  days  of  care  per  eligible. 
The  difference  between  these  two  rates  multiplied  by  the  eligible 
population  in  the  second  time  period  will  yield  an  estimate  of  the  total 
days  of  care  averted  through  the  concurrent  review  process  in  each  area. 

Information  on  the  daily  charge  for  room  and  board  will  be  collected  for 
each  hospital  in  the  PSRO  area  in  order  to  compute  a  weighted  average 
daily  room  and  board  charge  for  the  area.    (The  weights  to  be  used  will 
be  the  total  annual  Medicube  days  of  care  for  each  hospital  in  the  second 
time  period.)    The  products  of  these  average  daily  charges  and  the  estimates 
of  days  of  care  averted,  summed  over  the  PSRO  areas  will  yield  an  estimate 
of  the  impact  on  Medicube  expenditures  of  changes  in  hospital  utilization 
induced  through  concurrent  review.    (Table  7,  Item  11) 

The  weighted  average  room  and  board  charge  is  used  on  the  assumption  that, 
in  the  absence  of  PSRO  activities,  the  estimated  days  of  care  averted 
could  have  been  provided  in  any  of  an  area's  hospitals.    The  estimated 
"Savings"  therefore  must  be  based  on  each  hospital's  Medicube  volume. 
The  estimate  will  not  reflect,  however,  the  impact  on  expenditures  of  in- 
creases in  the  intensity  of  services,  if  any,  that  are  induced  by  concurrent 
review. 

Net  Savings 

The  result  of  the  previous  analysis  will  be  supplemented  with  data  available 
from  the  PSRO  MIS  and  contract  reports  to  determine  whether  the  estimated 
benefits  of  concurrent  review  exceed  the  costs. 

Annual  concurrent  review  expenditures  (which  will  include  an  allocated 
portion  of  administrative  costs)  will  be  determined  for  each  PSRO  area  to 
be  used  in  the  computation  of  an  estimated  benefit-cost  ratio.    If  con- 
current review  has  been  an  efficient  system  these  ratios  should  be  greater 
than  one.    (Table  7,  Item  12) 

An  analysis  of  the  variations  in  these  benefit-cost  ratios  will  be  con- 
ducted to  identify  PSRO  characteristics  which  have  an  impact  on  the  efficiency 
of  concurrent  review.    The  estimated  benefit-cost  ratio  will  be  used  as 
the  dependent  variable  in  a  weighted  least  squares  regression  analysis. 
(The  weights  to  be  used  will  be  based  on  the  annual  number  of  Medicube  days 
of  care  in  each  area.)    Policy  variables  included  in  the  analysfs  are  the 
percentage  of  Medicube  discharges  treated  in  delegated  hospitals  and  con- 
current review  expenditures  per  Medicube  eligible.    Qualifying  variables 
include  the  number  of  hospitals  in  the  PSRO  area,  the  annual  number  of 
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Medicube  discharges,  and  dummy  variables  to  represent  the  PSRO's  cohort. 
The  area  wide  occupancy  rate  will  be  included  as  a  literature  variable 
as  will  the  ratio  of  extended  care  beds  to  the  population  older  than  65. 

The  estimated  coefficient  of  the  cohort  variable  will  be  of  particular 
interest.    If,  in  fact,  PSROs  reduce  most  or  all  unnecessary  utilization 
in  their  beginning  years,  an  estimated  benefit-cost  ratio  based  on  changes 
in  utilization  will  decline  over  time.    A  PSRO  may  function  well,  however, 
and  exhibit  a  relatively  low  utilization  rate  rather  than  a  large  decrease 
in  the  utilization  rate.    Indeed,  in  the  long  run,  the  effect  of  con- 
current review  should  be  to  maintain  such  relatively  low  utilization  rates. 
In  particular,  special  attention  must  be  given  to  the  first  cohort  of 
conditional  PSROs.    Many  were  experienced  review  organizations  prior  to 
becoming  PSROs,  and  may  have  already  achieved  their  impact. 

2.  Comments 

a.    The  assumption  implicit  in  the  analyses  described  above  is 
that  a  change  in  admission  rates  can  be  used  as  the  measure 
of  AC  impact.    There  are  a  number  of  other  factors  whose  effects 
could  be  measured  in  order  to  assess  the  impact  of  the  AC 
mechanism  more  accurately. 

One  of  these  factors  is  the  denial  rate.    In  an  effective 
system,  the  denial  rate  will  be  smaller  than  the  change  in 
admission  rates  and  should  decline  gradually  to  some  small 
percentage  of  total  reviews.    In  an  ineffective  system  the 
denial  rate  could  well  be  greater  than  the  change  in  admission 
rates  as  admissions  are  merely  shifted  among  hospitals.  In 
either  case,  an  increase  in  the  denial  rate  could  be  cause  for 
concern. 

Unfortunately,  even  declines  in  certain  diagnostic  specific 
admission  rates  are  not  unambiguous  indicators  of  AC 
effectiveness.    The  AC  process  may  be  evaded  through  changes  in 
diagnosis  status  once  the  patient  has  passed  through  admission 
certification.    Evidence  that  the  AC  process  is  being  evaded, 
as  measured  by  a  substantial  increase  in  the  percentage  of 
discharges  with  a  different  admitting  diagnosis  will  indicate 
that  particular  PSROs,  institutions,  or  practitioners  need 
to  be  monitored  more  closely. 

A  third  factor  relevant  to  the  interpretation  of  AC  effectiveness 
is  the  percentage  of  admissions  granted  certification  as  an 
exception.    The  potential  effect  of  AC  may  be  invalidated  if 
admitting  physicians  are  able  to  consistently  override  criteria 
for  admission.    The  percentage  of  cases  of  medically  necessary 
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hospitalizations  which  do  not  meet  admission  criteria 
should  be  stable.    Increases  in  that  percentage  may 
signal  another  form  of  evasion  of  the  AC  process. 

b.  Another  consideration  in  the  interpretation  of  changes 
in  admission  rates  is  the  expansion  of  HMO's  and  other 
prepaid  health  organizations.    If  hospital  utilization  is 
indeed  lower  for  members  of  prepaid  organizations,  a  switch 
to  greater  enrollment  in  such  organizations  should  be 
accompanied  by  a  decrease  in  admission  rates.    Marked  changes 
in  prepaid  enrollment  by  the  Medicube  population  may  invalidate 
deductions  about  AC  effectiveness  based  on  simple  changes  in 
admission  rates. 

c.  The  assumption  implicit  in  many  of  the  analyses  is  that  con- 
current review  will  affect  all  diagnostic  categories  on  the 
average  as  they  do  the  sample  categories.    This  assumption  is 
more  suspect  on  a  local  level  than  on  the  national  level  and 
various  adjustments  must  be  made  for  analyses  of  intra-PSRO 
variations.    For  example,  when  an  individual  PSRO  or  institution 
bases  its  concurrent  review  program  on  a  select  set  of  diagnoses, 
the  diagnoses  studied  for  the  concurrent  review  analyses  must 

be  considered  a  subset  of  the  select  set  rather  than  a  subset 
of  all  diagnoses. 

d.  The  analyses  based  on  readmissions  may  be  difficult  to 
accomplish  in  certain  institutions  because  the  readmissions 
rates  are  so  low.    There  are  a  number  of  ways  of  dealing  with 
this  difficulty.    One  way  is  to  perform  the  analyses  over 
longer  time  periods  allowing  the  cases  to  accumulate.    It  was 
with  this  in  mind  that  the  analyses  have  been  designed  to  be 
accomplished  using  semi-annual  observations  rather  than  quarterly 
ones.    Instead,  it  might  be  necessary  for  the  analyses  to  be 
based  on  annual  observations  to  allow  for  a  sufficient  number 

of  cases  to  accumulate. 

Another  method  for  obviating  this  difficulty  is  to  aggregate 
data  across  any  of  its  cross-tabulable  variables.    Thus,  all 
prior  lengths  of  stay  can  be  combined  into  one  group  (rather 
than  keeping  early  discharges,  ontime  discharges  and  extended 
stay  discharges  separately)  as  can  data  for  more  than  one 
diagnostic  category,  more  than  one  hospital,  etc.    The  problem 
in  doing  this  is  that  the  data  cannot  then  be  analyzed  for 
differences  between  the  categories  within  these  factors. 

e.  CSR  consists  of  decisionmaking  tasks.    When  decisionmaking 
occurs,  there  is,  of  course,  a  chance  for  error.    Two  kinds  of 
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error  can  occur:  Cal  patients  who  do  not  need  to  have 
stays  extended  are  nevertheless  retained  in  the  hospital  or 
(b)  patients  who  should  be  retained  in  hospitals  are  dis- 
charged.   Should  CSR  operate  defectively  in  committing  the 
first  error,  there  is  not  way  of  detecting  it  and  the  case 
will  not  come  to  anyone's  attention  as  being  a  review  error. 
When  the  second  type  of  error  is  committed,  however,  there  are 
some  indications  that  an  error  was  made.    For  example,  if  the 
patient  does  not  revert  to  ambulatory  care,  there  is  no  way 
of  detecting  the  error,  but  if  he  does,  either  (a)  the  later 
judgment  of  the  ambulatory  care  physician  or  (b)  the  length 
of  time  that  ambulatory  care  is  necessary,  may  provide  in- 
dications of  a  premature  discharge-    To  make  later  judgments 
meaningful  would  probably  require  a  mechanism  to  make  sure  that 
the  in-patient  attending  physician  and  the  post- institutional 
ambulatory  care  physician  are  not  the  same  person.  Some 
attempts  at  seeing  whether  the  measurement  of  these  errors  is 
possible  is  properly  relegated  to  the  status  of  a  research 
question.    Whether  lengthy  post-institutional  ambulatory  care 
may  also  be  used  for  detecting  these  errors  is  also  properly  a 
research  question.    If  research  results  in  a  positive  finding, 
some  means  of  extracting  this  information  from  both  out-patient 
clinic  and  private  physician  practice  ambulatory  care  files  shoul 
be  incorporated  into  later  versions  of  this  evaluation  strategy 
which  include  review  of  ambulatory  care. 

The  advance  designation  of  the  diagnoses  to  be  used  in  these 
analyses  may  induce  a  Hawthorne  effect.    For  example,  extra 
time  and  effort  may  be  devoted  to  monitoring  utilization  in 
these  diagnostic  categories.    As  a  result,  it  may  be  necessary 
to  analyze  utilization  changes  involving  a  small  set  of 
additional  diagnoses  which  will  not  be  designated  at  this  time. 
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Footnotes  -  Table  7 


1/   The  assumption  underlying  the  quasi-experimental  analyses  is  that 
the  impact  of  PSRO  activities  can  be  estimated  as  the  difference 
between  the  annual  (proportional)  changes  in  utilization  observed 
in  the  PSRO  active  areas  and  those  changes  observed  in  non- 
active  areas.     These  differences  must  be  statistically  significant 
for  the  rest  of  the  analysis  to  be  meaningful.    The  standard 
deviations  of  these  estimates  are  presented  as  Xq  and  Yq. 

The  analyses  involving  items  1  through  4  rely  upon  a  separation 
of  PSRO  impact  into  two  components:    one  due  to  changes  in  admission 
rates  and  one  due  to  changes  in  lengths  of  stay.     The  separation 
derives  from  a  mathematical  relationship  between  the  product  of 
two  factors  and  the  two  individual  factors,  vi  z . ,  the  proportional 
change  in  the  former  is  equal  to  the  sum  of  the  proportional 
changes  in  the  latter. 

The  attribution  to  AC  activities  of  the  component  due  to  changes 
in  admission  rates  relies  upon  the  mild  assumption  that  CSR 
activities  will  have  little  or  no  impact  on  admission  rates.  The 
attribution  solely  to  CSR  of  the  component  due  to  changes  in  lengths 
of  stay  is  a  stronger  assumption,  however  it  is  incorrect  only 
to  the  extent  that  AC  activities  more  than  proportionately  reduce 
the  incidence  of  relatively  short  lengths  of  stay.    To  that 
extent  the  measure  of  AC  impact  will  be  biased  upward  and  that 
of  CSR  biased  downward.    The  biases,  however,  differ  only  in 
sign  so  that  the  overall  estimate  of  impact  should  not  be  a  biased 
one. 

2/    Estimation  of  the  days  of  care  impact  of  concurrent  review  requires 
a  comparison  involving  two  mutually  exclusive  states  of  the  world. 
Utilization  in  an  active  PSRO  area  must  be  compared  to  the  utilization 
that  would  have  occurred  in  the  same  area  in  the  absence  of  PSRO 
activities.    The  latter,  obviously,  can  not  be  observed.    It  can 
be  estimated,  however,  on  the  assumption  that  it  differs  from  the 
observed  utilization  by  the  proportions  x  and  y  (Items  1  and  2). 
The  days  of  care  impacts  are  thus  functions  of  x,  y,  and  the 
observed  utilization,  as  specified  in  Items  3  and  4. 
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Issue  #2:    IS  CONCURRENT  QUALITY  ASSESSMENT  (CQA)  -  AS  CARRIED  OUT  IN 
THE  PSRO  PROGRAM  AS  A  FORM  OF  CONTINUED  STAY  REVIEW  -  AN 
EFFECTIVE  METHOD  OF  ASSURING  THE  APPROPRIATE  UTILIZATION  OF 
SERVICES  (DIAGNOSTIC  AND  THERAPEUTIC  PROCEDURES)? 


A.  Sub-issues 

1.  To  what  extent  has  CQA  (focused  on  assuring  compliance  with 
"essential"  criteria  elements)  resulted  1n  improvements  in 
patient  outcomes? 

2.  To  what  extent  has  CQA  resulted  in  a  reduction  of  deviations 
from  criteria  and  standards? 

3.  What  is  the  cost  of  CQA? 
Rationale: 

One  variant  of  continued  stay  review  1s  a  detailed  assessment  of  the 
quality  of  care  provided  to  Medicube  beneficiaries.    It  is  expected 
that,  where  adopted,  CQA  will  take  the  form  of  an  evaluation  of  patient 
management  with  respect  to  a  particular  set  of  criteria  elements.  Under 
such  a  procedure,  appropriate  utilization  of  services  will  be  defined 
by  the  scope  of  the  locally  adopted  criteria  sets  used  for  CQA. 

CQA  itself,  as  a  quality  assurance  mechanism,  1s  Important  because 
of  the  widespread  belief  that  assuring  compliance  with  selected 
criteria  elements  will  have  a  beneficial  impact  on  patient  outcomes. 
It  will  be  important  for  the  PSRO  program  to  determine  the  extent  to 
which  such  a  relationship  exists. 

Few  PSROs  are  expected  to  initiate  CQA  activities  in  the  program's 
early  stages,  but  it  will  be  important  to  assess  these  efforts  to 
ascertain  whether  CQA  can  be  an  effective  mechanism  to  assure 
compliance  with  the  criteria. 

B.  Management/ Pol  icy  Decisions 

The  analyses  of  this  issue  will  provide  essential  information  for 
decisions  to: 

1.  Change  policy  with  respect  to  the  continued  and/or 
increased  promulgation  of  CQA, 

2.  Advocate  changes  1n  the  type  of  criteria  used  1n  CQA, 
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C.  Analysis 

The  analyses  of  the  sub-issues  in  this  area  will  be  based  on  information 
derived  from  a  subset  of  the  PSROs  that  initiate  CQA  activities.  The 
number  of  such  PSROs  is  expected  to  be  quite  small.    Foremost  among 
them  will  be  the  PSROs  participating  in  the  experiments  of  the  Private 
Initiative  in  PSRO  (The  Kellogg  Foundation  Study).    These  PSROs  will 
be  the  primary  source  of  data  on  the  effect  of  CQA  on  patient  outcomes. 
In  addition  to  such  information,  the  Private  Initiative  PSROs  will 
provide  information  on  compliance  with  the  criteria  and  on  the  cost  of 
CQA.    These  latter  two  types  of  information  will  also  be  obtained  from 
all  other  PSROs  which  perform  CQA  activities  and  which  are  surveyed  for 
these  analyses. 

Health  Outcomes 

The  Private  Initiative  experiments  will  be  conducted  in  five  PSRO  areas 
during  calender  1975  and  will  include  68-80  hospitals.    Certain  hospitals 
from  this  group  will  initiate  CQA  activities,  while  some  of  the  others 
will  act  as  "CQA  Controls."    The  medical  staffs  of  the  CQA  hospitals 
will  adopt  the  policy  of  requiring  compliance  with  essential  criteria 
for  seven  selected  diagnoses: 


S 

(c 
(d 
(e 
(f 

(g 


Acute  myocardial  infarction, 
Bacterial  pneumonia, 
Bacterial  urinary  tract  infection, 
Acute  gastroenteritis  in  children 
Acute  upper  gastrointestinal  bleeding, 
Acute  appendicitis,  and 
Cholecystitis  and  Cholelithiasis. 


In  addition  to  information  on  patient  management  compliance  with  the 
essential  criteria,  information  on  the  immediate  outcomes  of  hospital 
care  will  be  collected  for  each  Medicube  patient  hospitalized  for  any 
one  of  the  seven  diagnoses.    To  be  included  are  data  on  preventable 
complications  not  prevented  and  clinical  and  functional  status  at 
discharge.    Finally,  patient  information  will  include  an  indication  of 
whether  or  not  there  was  a  readmission  to  a  hospital  for  the  same  com- 
plaint within  90  days  of  the  original  discharge.    These  outcome  measures 
will  be  converted  to  incidence  measures,  i.e.,  percentage  of  cases 
where  patient  is  discharged  with  functional  status  equal  to  or  better 
than  a  threshold  status  chosen  by  the  Initiative  designers.  These 
incidence  measures  will  form  the  bases  for  comparisons  between  the 
CQA  hospitals  and  the  controls,    (Table  8,  Item  1)    If  CQA  has  a 
beneficial  impact  on  patient  outcomes,  the  CQA  hospitals  should  have 
a  significantly  higher  percentage  of  cases  discharged  at  threshold 
status  or  higher,  and  should  have  significantly  lower  percentages  of 
both  (a)  cases  with  preventable  complications  that  have  not  been  prevented 
and  (b)  readmissions  within  90  days. 
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Compliance  with  Criteria  and  Standards 

These  analyses  will  be  completed  in  calendar  year  1977,  Compliance 
with  the  criteria  will  be  measured  in  terms  of  deviations  from  the 
criteria  and  standards.    The  measure  of  deviation  will  be  the 
percentage  of  cases  at  variance  from  the  criteria  and  standards  rather 
than  a  summed  magnitude  of  deviation.    A  more  sophisticated  analysis, 
using  summed  magnitudes  of  deviations,  would  require  a  methodology  for 
weighting  deviations  based  on  both  individual  criteria  elements  and 
distinct  criteria  sets.    Such  methodology  is  not  available  at  this  time. 

It  is  important  to  note  at  this  point  that  the  percentage  of  deviant 
cases  will  rarely,  if  ever,  be  zero.    This  is  the  case  if  only  because 
of  normal  variation  in  (a)  the  general  health  of  patients,  (b)  the  severity 
of  their  illnesses,  and  (c)  the  timing  of  their  entry  into  the  medical 
system  with  respect  to  the  course  of  their  disease.    In  fact,  if  there 
are  no  cases  that  are  reported  which  are  not  in  compliance  with  a  par- 
ticular criteria  set,  that  criteria  set  should  be  immediately  suspect. 
The  percentage  of  deviant  cases  should  be  small,  however,  in  an  effective 
system,  a  reduction  in  the  percentage  of  cases  not  1n  compliance  with 
the  criteria  will  be  an  indicator  of  an  effective  CQA  system. 

Criteria  sets  with  more  than  one  element  present  a  measurement  problem 
in  that  not  all  criteria  elements  are  of  equal  importance.  That 
problem  will  be  addressed  as  follows: 

(a)  Where  CQA  system  criteria  are  based  on  so-called  critical  or 
essential  criteria  elements,  non-compliance  with  any  one  of 
the  elements  is  sufficient  to  designate  patient  management  as 
deviant  from  criteria  and  standards, 

(b)  Where  a  criteria  set  does  not  designate  critical  elements, 
local  PSRO  judgment  will  be  used  to  designate  a  minimum 
compliance  level  in  terms  of  the  percentage  of  criteria 
elements  which  must  be  complied  with  for  appropriate  patient 
management.    Cases  not  meeting  that  level  will  be  designated 
as  deviant. 

(c)  PSROs  which  adopt  systems  with  weights  assigned  to  each 
criteria  element  will  convert  compliance  measures  to  percentages 
by  computing  the  ratio  of  actual  compliance  to  potential 

compl iance. 

Reductions  in  deviations  from  criteria  and  standards  will  be  measured 
by  comparisons  to  matched  institutions  without  CQA  or  by  comparisons  to 
prior  time  periods.    (Table  8,  Item  2)    The  former  approach  will  be 
used  in  the  case  of  the  Private  Initiative  experiments,  i.e.,  the  CQA 
hospitals  will  be  compared  to  the  controls  in  terms  of  the  percentages 
of  non-compliant  cases. 
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Comparisons  to  prior  periods  will  be  difficult,  because  only  in  a  very 
select  number  of  well  established  review  organizations  does  there 
exist  baseline  data.    This  problem  will  be  addressed  by  either  of  two 
approaches.    The  first  will  be  to  perform  an  MCE  on  the  particular 
diagnosis/or  problem  for  which  CQA  activities  are  planned  prior  to  the 
initiation  of  CQA.    That  MCE  will  utilize  the  planned  CQA  criteria  and 
standards,  and  should  be  based  on  the  patient  management  experience 
(of  no  less  than  six  months  duration)  immediately  prior  to  the  initiation 
of  CQA  activities,    The  MCE  results  will  be  interpreted  in  terms  of  the 
percentage  of  cases  studied  that  were  found  not  1n  compliance  with  the 
planned  CQA  criteria  and  standards,  and  reductions  1n  deviations  will 
be  measured  by  comparison  to  those  results.    This  approach  has  the 
advantage  that  1t  may  be  possible  to  confirm  that  CQA  activities  can  be 
of  potential  benefit  for  the  particular  diagnosis  or  problem  in  question. 
Where  this  approach  is  not  used,  the  first  six  months  of  CQA  for  a 
particular  diagnosis  problem  will  be  designated  as  the  baseline  period. 

Cost  of  CQA 

There  are  two  types  of  cost  statistics  associated  with  CQA:    the  cost 
of  providing  CQA  services  and  changes  in  the  cost  of  medical  care 
due  to  utilization  changes  induced  by  CQA.    (An  analysis  of  the  latter 
is  described  in  the  research  section  of  Summative  Issue  5.)    The  former 
will  include  the  costs  of  criteria  development  for  CQA  and  CQA  review 
coordinator  and  other  personnel  costs  and  associated  expenses,  but  not 
the  cost  of  any  preparatory  MCEs,    These  costs  should  be  normalized 
by  dividing  the  cost  of  CQA  for  a  particular  diagnosis/problem  by  the 
number  of  discharges  with  that  diagnosis/problem  which  were  actually 
reviewed  with  respect  to  the  CQA  criteria  and  standards.    (Table  8, 
Item  3) 
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Issue  #3:  IS  RETROSPECTIVE  HEALTH  CARE  REVIEW,  AS  CARRIED  OUT  IN  THE 
PSRO  PROGRAM,  AN  EFFECTIVE  METHOD  OF  ASSESSING  THE  QUALITY 
AND  APPROPRIATENESS  OF  THE  UTILIZATION  OF  SERVICES? 


A.  Sub-Issues 

1.  How  effective  are  the  methods  employed  by  the  PSROs  (including 
institutional,  practitioner  and  patient  profiling)  for 
identifying  problem  areas  that  require  further  assessment  by 
MCE  studies? 

2.  Are  adequate  methodologies  being  utilized  in  the  performance 
of  MCE  studies? 

3.  To  what  extent  have  MCE  studies  effected  corrective  action? 

4.  Are  some  MCE  models  more  effective  and  do  they  cost  less  to 
implement  than  others? 

Rationale: 

The  purposes  of  MCE  studies  are  to  detect  and  correct  conditions  that 
are  resulting  in  reduced  quality  of  health  care  and/or  inappropriate 
utilization  of  health  care  services  delivered  under  Medicube  programs. 
To  achieve  these  objectives,  each  MCE  should  include  the  following 
steps: 

1.  Problem  Detection.    Use  patient,  physician,  or  institutional 
profiles,  other  review  activities,  or  the  judgment  of  the 
MCE  committee  and  other  medical  care  personnel  to  detect 
the  problems  that  should  be  subjected  to  MCE  study. 

2.  Problem  Selection.  The  problems  detected  should  be  weighted 
so  that  the  most  important,  in  terms  of  loeal  impact  (health 
outcome  and  cost),  are  chosen  for  study. 

3.  Study  Design.    The  MCE  study  should  be  designed  so  that  it  is 
methodologically  sound,  free  of  bias,  and  the  results  can  be 
used  in  proposing  solutions,  if  necessary,  to  medical  or 
administrative  procedures  that  the  MCE  study  demonstrations 
are  in  need  of  correction.!/ 


V    The  conduct  of  the  study  may  require  the  utilization  of  criteria  and 
standards  which,  in  most  cases,  should  have  already  been  formulated 
as  a  result  of  other  PSRO  activities.    If  such  criteria  and  standards 
are  not  already  available,  the  additional  step  of  their  formulation 

must  be  added  before  this  step  is  undertaken. 
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4.     Conduct  of  Study.    The  MCE  study  should  be  carried  out  in 
consonance  with  the  study  design. 

5„     Analysis  of  results/Suggestions  for  Change.    The  results 

should  be  analyzed  and  reported.    The  analysis  should  include 
suggestions,  where  appropriate,  for  changes  in  medical  and/or 
administrative  procedures  that  will  improve  the  quality  of 
patient  care,  and/ or  the  effective  and  efficient  use  of 
services. 

6.     Change  Implementation.  U   Where  appropriate,  changes  should 
be  instituted. 

7o     Evaluation  of  Change.    A  follow-up  study  should  be  con- 
ducted to  determine  whether,  and  to  what  extent,  the  changes 
have  had  the  desired  effects.    The  evaluation  itself  might 
consist  of  a  re-run  of  Steps  4  and  5  above  after  the  change 
has  been  under  implementation  for  a  long  enough  period  of 
time  to  have  had  an  effect. 


B.    Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for 
decisions  to: 

1.  Modify  the  regulations  that  deal  with  the  way  MCEs  are  to  be 
conducted; 

2.  Offer  additional  technical  assistance  in  the  conduct  of  MCEs; 

3.  Conduct  additional  special  studies; 

4.  Disseminate  some  MCE  findings  amongst  all  PSROs; 


C.  Analysis 

The  evaluation  of  this  issue  requires  a  technology  in  health  outcome 
assessment  which  currently  is  not  sufficiently  advanced  to  allow 
measurement  of  effectiveness  in  many  instances.    The  following  analytical 
methods,    therefore,  may  not  be  adequate  for  the  evaluation  of  MCE 


2/  The  word  "change"  will  be  used  throughout  the  discussion  of  this 
issue.  It  refers  to  structural  or  procedural  changes  or  changes 
in  continuing  education  that  are  recommended  as  the  result  of  an 
MCE  study. 
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studies.    They  do,  however,  represent  the  state-of-the-art  in  evalua- 
tion and  should  be  reassessed  as  to  their  merit  after  an  initial 
appl i cat ion. 

The  sub-issues  outlined  above  divide  themselves  over  the  seven  steps 
required  for  an  MCE  study.    The  measurement  of  how  effectively  and 
efficiently  these  steps  are  accomplished,  then,  provides  the  basis  for 
the  analysis  of  this  issue. 

Since  there  are  many  cooperative  arrangements  under  which  MCEs  are 
allowed  to  be  conducted,  it  is  difficult  to  predict  the  total  number 
of  MCEs  per  year  that  will  be  carried  out.    The  number  expected, 
however,  is  well  over  one  thousand  per  year  when  the  program  is  fully 
implemented.    Such  a  large  number  of  studies  would  make  a  detailed 
evaluation  of  each  study  prohibitively  expensive  and  time  consuming. 
On  the  other  hand,  to  avoid  examining  some  MCEs  in  a  detailed  fashion, 
would  result  in  leaving  several  important  evaluation  questions 
unanswered.    It  is  therefore  necessary  to  use  a  combined  approach 
in  which  (a)  the  processes  being  used  in  all  MCEs  are  characterized  and 
compared  with  the  objectives  of  the  program,  and  (b)  a  detailed 
evaluation  of  a  sample  of  MCEs  is  carried  out. 

The  first  of  these  approaches  will  consist  of  analyzing  data  provided 
through  the  PMIS,  namely: 

(a)  An  MCE  study  abstract  describing  the  characteristics  of  the 
basic  MCE  study,  its  results  and  recommended  changes; 

(b)  An  MCE  Restudy  Report  describing  the  changes,  if  any,  that 
have  occurred  since  the  MCE  study  was  carried  out;  and 

(c)  An  MCE  Study  Status  Report  describing  each  PSRO's  MCE  study 
program  by  listing  the  studies  and  their  state  of  completion. 

This  approach  will  be  used  to  (1)  evaluate  the  problem  identification 
and  problem  selection  process,  and  (2)  characterize  the  MCE  studies 
with  respect  to  the  variety  of  techniques  utilized  and  problems 
addressed. 

The  second  approach  will  be  pursued  through  site  visits.    These  visits 
will  utilize  the  information  maintained  by  PSROs  to  document  the  MCEs 
as  well  as  that  gained  from  interviews  with  those  PSRO  and  institutional 
staff  that  were  involved  in  the  MCE  studies. 

The  purposes  of  the  site  visits  are  twofold: 

(a)     To  evaluate  elements  of  MCEs  (e.g.,  methodology)  that  cannot 
be  evaluated  through  abstracts; 
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(b)     To  verify  that  the  MCE  study  and  restudy  data  being  placed 

into  the  PMIS  are  accurate  (and  to  understand  the  inaccuracies 
if  there  are  any)- 

Effectiveness  of  Problem  Identification  and  Selection 

This  sub-issue  will  be  evaluated  in  three  steps  on  an  annual  basis. 

The  first  step  will  consist  of  the  preparation  of  frequency  distributions 

of  the  following  information: 

(a)  Problems  studied  -  a  schema  for  classifying  MCE  topics  and 
descriptions  into  problem  categories  will  be  developed  after 
some  experience  with  the  types  of  problems  that  are  chosen 
for  study.    It  is  anticipated  that  after  the  schema  is 
developed,  MCEs  will  be  reported  with  the  categories  already 
coded  on  the  abstract  forms. 

(b)  Specialties  and/or  departments  -  a  schema  will  be  developed 
for  classifying  and  coding  studies  by  the  specialties  or 
departments  involved. 

(c)  Problem  source  -  the  categories  for  this  variable  will  be: 

.    profile  analysis  .    high  volume  of  incidence 

.    concurrent  review  .    perceived  need  by  staff 

.    other  MCE  study  .    prearranged  PSRO  plan 

.    analysis  of  medical  .    other  (specify) 

records 

(d)  Study  scope  -  The  categories  for  this  variable  are: 


.  national 
.  regional 

.    PSRO-wide  (by  type) 
.  State-wide 
.    multiple  county 

.    other  (single  county,  single  city, 
or  partial  city) 

.    delegated  hospitals 
.    multiple  institutions 
.    single  institutions 

These  distributions  will  be  used  to  detect  patterns  with  respect  to 
problems,  sources,  specialties/departments,  and  scopes.    The  results  of 
this  analysis  will  be  used  to  redirect  MCE  study  efforts.    (Table  9, 
Item  1  ). 
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The  second  step  will  consist  of  a  potential  effectiveness  criteria 
comparison  with  the  problems  being  studied,    To  perform  this  study, 
a  random  sample  of  no  less  than  10%  of  all  MCE  study  abstracts 
submitted  during  one  year  will  be  selected,    This  sample  of  abstracts 
will  be  reviewed  by  a  physician  panel.    The  panel  will  be  constituted 
of  members  with  a  great  diversity  in  the  geographical  and  institutional 
settings  in  which  they  gained  their  clinical  experience.  Using 
four  separate  scales,  one  for  each  of  the  criteria,  each  member  of 
the  panel  will  rate  each  abstract  in  the  sample.    The  four  criteria 
are: 

(a)  Potential  -  the  problem  is  in  an  area  where  change  is 
possible. 

(b)  Morbidity  -  the  degree  to  which  the  study  addressed  a 
problem,  which  if  solved,  would  decrease  the  seriousness 
of  an  illness. 

(c)  Cost  -  the  actual  or  estimated  changes  in  costs  that  could 
result  if  some  type  of  change  in  procedures,  practices, 

or  organization  were  instituted,  or  additional  equipment 
or  physical  plant  acquired, 

(d)  Frequency  of  Occurrence  -  the  number  of  admissions  affected 
or  the  total  number  of  times  an  event  occurs  or  a  diagnostic 
or  therapeutical  procedure  is  performed. 

It  may  be  necessary  to  augment  the  abstract  data  with  survey  data 

on  the  last  two  of  these  criteria  to  assure  that  accurate  and  sufficient 

information  is  available  to  the  panel. 

The  rating  procedure  employed  will  not  require  that  every  study 
satisfy  all  four  criteria  in  order  to  receive  a  high  effectiveness 
rating.    Also,  it  is  recognized  that  there  is  some  inherent  error 
in  having  physicians  rate  the  importance  of  an  MCE  by  reviewing  the 
problem  under  study  with  a  study  abstract.    Individual  studies, 
however,  are  not  the  focus  of  this  evaluation  issue  and  there  is  no 
intent  to  aggregate  the  ratings  on  an  individual  hospital  or  PSRO 
basis.    Rather,  the  purpose  of  this  issue  is  to  detect  patterns 
that  are  emerging  as  well  as  to  arrive  at  some  overall  assessment 
of  the  MCE  identification  and  selection  process  on  a  national, 
program-wide  basis.    By  using  a  pansl  and  aggregating  the  data 
across  all  PSROs,  small  rating  errors  can  be  expected  to  wash  out. 

To  determine  the  overal 1  effectiveness  of  problem  identification  and 
selection  for  the  MCE  studies  performed  during  the  evaluation  period, 
then,  two  measures  will  be  computed.    The  first  measure  is  the  average 
of  the  individual  topic  ratings  and  the  second,  the  ratio  of  the 
number  of  MCE  studies  resulting  in  some  corrective  action  being  taken 
to  the  total  number  of  MCE  studies.    (Table  9,  Item  1) 
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The  third  step  will  consist  of  using  the  ratings  derived  under  step 
two  above,  to  evaluate  problem  sources.    Mean  ratings  will  be  computed 
for  each  problem  source,  and  these  means  compared  to  determined  the 
quality  of  the  MCE  studies  generated  by  each  source.    (Table  9,  Item  le) 

Methodological  Adequacy 

During  the  early  phases  of  each  PSRO's  development  their  initial 
concern  will  be  to  get  some  MCEs  started.    Because  these  studies  may 
represent  initial  PSRO  and  hospital  endeavors  in  retrospective  studies, 
the  level  of  methodological  design  can  be  expected  to  vary  from  PSRO 
to  PSRO.    The  purpose  of  this  step  is  to  assess  the  degree  to  which 
the  study  designs  reflect  the  need  for  additional  technical  assistance, 
as  well  as  determine  what  the  nature  of  that  technical  assistance  ought 
to  be.   To  the  extent  that  it  may  also  reflect  the  difficulties  that 
PSROs  may  be  having  in  carrying  out  the  studies  under  existing  regula- 
tions, it  will  also  provide  guidance  as  to  how  those  regulations  should 
be  modified.    Again,  the  intent  of  this  issue  analysis  is  not  to 
evaluate  the  individual  PSRO,  and  the  analysis  will  be  based  on  an 
aggregation  of  the  data  at  a  national,  program-wide  level.    A  secondary 
purpose  of  this  issue  analysis  is  to  verify  the  accuracy  of  the  MCE 
part  of  the  PMIS  data  base. 

To  perform  this  analysis  it  will  be  necessary  to  conduct  on-site  visits. 
In  some  instances  the  studies  under  review  will  be  those  that  were 
conducted  by  individual  hospitals.    In  those  cases,  all  the  data  that  are 
needed  for  this  analysis  will  be  collected  by  working  through  the  PSRO. 

A  sample  of  PSROs  will  be  chosen  for  biannual  site  visits.  During 
the  site  visits  the  following  two  steps  will  be  undertaken. 

(a)    Examination  of  MCE  Procedures 

Through  the  examination  of  MCE  documentation  and  interviews 
with  PSRO  and  institutional  staffs,  the  procedures  used 
for  the  conduct  of  MCEs  will  be  evaluated.    Since  the 
evaluation  will  be  conducted  on  MCEs  with  a  wide  diversity 
of  subject  matter  and  methodological  approaches,  the 
evaluation  methodology  must  be  based  on  judgments.  The 
types  of  judgments  which  must  be  made  require  expertise 
from  persons  with  a  number  of  different  technical  backgrounds. 
Therefore  a  site  visit  panel  will  be  formed  for  conductinq 

these  evaluations.    The  panel  will  be  composed  of  experimental 
methodologists/statistical  analysts,  medical  records 
specialists,  hospital  administrators  and  clinicians. 
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(b)    Data  Verification 

A  sample  of  MCEs  will  be  chosen  from  each  of  the  sites 
visited.    The  abstract  and  restudy  reports  will  be  compared 
to  the  on-site  study  documentation  to  determine  how  accurately 
they  represent  the  MCE  study  that  was  reported.    The  outcome 
of  this  task  may  either  be  (1)  suggestions  for  modification 
of  the  forms,  instructions  and  reporting  procedures  if 
inaccuracies  are  due  to  mi scommuni cation,  or  (2)  suggestions 
for  increased  technical  assistance  and  monitoring  if  the 
inaccuracies  are  due  to  inabilities  to  carry  out  MCE  studies 
properly. 

Since  the  evaluation  of  this  issue  is  to  be  (a)  accomplished  by  means 
of  special  studies,  (b)  carried  out  by  teams  of  personnel  who  will 
have  to  provide  the  detailed  designs  of  the  evaluations  they  will 
perform,  and  (c)  based  on  MCE  studies  which  may  assume  a  variety  of 
subjects  and  methodologies,  only  general  guidelines  for  the  analysis  can 
be  included  in  this  plan.    Based  on  these  guidelines,  a  plan  for  the 
evaluation  of  this  issue  will  be  developed  detailing  the  procedures  to 
be  used  during  the  evaluation  period.    The  plan  should  consist  of  at 
least  the  following  elements: 

a,  Number  and  qualifications  of  personnel  comprising  the  evalua- 
tion teams; 

b.  Designation  of  the  PSRO  sites  to  be  visited; 

c-    General  procedures  to  be  followed  by  the  evaluation  teams, 
including  the  way  that  either  PSROs  or  MCEs  are  to  be 
sampled  for  evaluation  purposes;  and 

d.    The  nature  of  the  report  to  be  made  by  the  evaluation  teams. 
During: the  site  visits  the  elements  of  the  MCE  design  that 
will  be  reviewed  are: 


a. 

The  measurements  (e.g 
are  taken; 

. ,  the  dependent  variables)  that 

b. 

The  sample  size; 

c. 

The  sampling  method; 

d. 

The  controls  that  are 

utilized; 

e. 

Potential  biases; 
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f.      The  manner  in  which  the  data  were  reduced  and  analyzed. 

The  same  type  of  experts  can  review  the  procedures  used  for  the  study. 
The  factors  to  be  considered  here  are: 

a.  Personnel  involved  in  the  study  and  how  they  are  used. 

b.  Source  of  the  data. 

c.  Measurement  instrument/ records. 

It  should  be  pointed  out  that  MCEs  are  not  intended  to  be  laboratory 
controlled  studies  and  in  reviewing  the  MCE  studies  the  panel  is  not 
expected  to  use  journal  level  criteria  in  assessing  the  methodological 
approaches  used.    Rather,  they  will  be  provided  with  guidance  to  make 
their  assessments  on  the  basis  of  whether  the  design  of  the  study 
would  have  reasonably  permitted  the  drawing  of  the  intended  conclusions. 
(Table  9,  Item  2b) . 


Effectiveness  of  MCE's  in  bringing  about  change 

The  basis  for  this  analysis  will  be  a  sub-sample  of  those  MCE  studies 
completed  during  the  evaluation  period  that  have  resulted  in  some 
corrective  action.    MCE  study  abstracts  and  restudy  abstracts  will 
provide  the  data  source.    The  sample  will  be  stratified  by  PSRO 
characteristics  (e.g.,  size,  population  density,  and  region). 

For  each  study  in  this  sample,  the  corresponding  restudy  abstract  will 
be  rated  by  a  panel  of  experts  to  determine  the  extent  to  which  the 
recommended  corrective  action  has  been  successful  in  achieving  its 
objecti ve(s) .    For  studies  which  focus  on  changes  in  medical  practice, 
the  rating  scale  will  be  based  on  compliance  to  the  recommended 
corrective  action.    In  some  cases  this  assessment  may  require  communica- 
tion with  the  PSRO  in  order  to  obtain  sufficient  information  for  the 
judgment.    A  measure  of  the  effectiveness  of  MCE  studies  should 
consider  both  the  level  of  success  and  total  impact  (cost  changes  and 
patient  outcome)  of  the  studies.    Impact,  however,  cannot  be  adequately 
measured.    The  current  state-of-the-art  in  outcome  measurement 
methodology  is  primitive,  at  best.    Furthermore,  the  results  of 
recommended  and  implemented  corrective  actions  do  not  necessarily 
affect  only  inpatient  care.    The  effectiveness  measure  that  will  be  used 
is  the  ratio  of  the  number  of  MCE  studies  which  have  resulted  in  some 
corrective  action  (weighted  by  their  level  of  success)  to  the  total 
number  of  the  MCE  studies  which  have  resulted  in  some  corrective  action. 
(Table  9,  Item  3). 
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Effectiveness  and  Relative  Costs  of  MCE  Study  Models 

Because  the  PSRO  program  allows  considerable  flexibility  in  the  conduct 
of  MCE  studies,  it  is  highly  likely  that  a  number  of  different  MCE 
study  models  will  be  utilized  by  the  PSROs  and  hospitals.    To  evaluate 
the  effectiveness  of  these  models  the  following  steps  will  be  per- 
formed: 


1.  A  survey  of  the  PSROs  will  be  conducted  to  identify  the 
most  frequently  used  types  of  MCE  study  models.  These 
models  will  be  grouped  into  major  model  types. 

2.  For  each  major  type  MCE  model  identified,  a  sample  of 
studies  will  be  drawn  from  the  PSROs.    Priority  will  be 
given  to  studies  which  deal  with  the  small  problem  or  topic 
but  under  different  model  types.    This  approach  will 
greatly  strengthen  the  comparisons  between  major  model  types. 

3.  Using  the  same  methodology  as  described  in  the  effectiveness 
analyses  above,  each  MCE  study  selected  will  be  analyzed  to 
determine  the  level  of  success  achieved. 


4.  The  model  types  will  be  compared  on  the  basis  of  a  measure 
of  their  relative  effectiveness.    This  measure  will  be 
computed  for  each  model  type  as  the  ratio  of  the  number  of 
MCE  studies  (for  model  type)  weighted  by  their  level  of 
success  to  the  total  number  of  MCE  studies  using  that 
model  (Table  9,  Item  4) . 

5.  From  each  of  the  PSROs  which  are  chosen  to  represent  a 
model  type,  a  sufficient  level  of  effort  information  will  be 
gathered  to  form  a  cost  estimate  for  each  model  type.  The 
most  effective  model  types  that  result  from  the  analyses 
described  under  paragraphs  1-4  above  will  be  compared  to 
determine   whether  the  effectiveness  associated  with  those 
models  derive  in  part  from  any  unwarranted  cost  factors. 
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Issue  #4:    HOW  EFFECTIVE  ARE  THE  VARIOUS  CORRECTIVE  MECHANISMS  APPLIED 

TO  INSTITUTIONS  OR  PRACTITIONERS  FOR  CORRECTING  INAPPROPRIATE 
UTILIZATION  OF  SERVICES  AND/ OR  IMPROVING  THE  QUALITY  OF 
CARE? 


A.  Sub-issues 

1.  What  has  been  the  incidence  of  use  of  various  corrective 
mechanisms  relating  to  concurrent  review? 

2.  What  has  been  the  impact  of  concurrent  review  denials  and 
special  concurrent  review  requirements,  (e.g.,  preadmission 
certification),  on  the  incidence  of  inappropriate  utilization 
of  services? 

3.  Have  recommendations  by  PSROs  for  continuing  education  had 
beneficial  impacts  on  individual  physician  performance? 

4.  Have  recommendations  for  organizational  changes  in  hospitals 
(as  a  result  of  PSRO  activities)  led  to  a  decrease  in 
inappropriate  utilization  of  services  and/or  improvements  in 
quality? 

5.  What  has  been  the  incidence  of  section  1160  (b)  sanctions, 
and  what  has  been  the  impact  of  such  sanctions? 

Rationale: 

It  is  the  function  of  a  PSRO  not  merely  to  assess  the  appropriateness 
of  the  utilization  of  health  services  and  the  quality  of  these  services 
but  also  to  use  its  authority  and  influence  to  assure  that  health 
services  are  appropriately  utilized  and  are  of  sufficient  quality. 

While  it  is  expected  that  much  of  the  PSRO  corrective  impact  will  be 
achieved  through  informal  (and  unrecorded)  physician  contacts      there  are 
available  to  PSROs  a  number  of  formal  corrective  mechanisms.  These 
include  : 

(a)  concurrent  review  denials; 

(b)  special  review  requirements  such  as  preadmission  certification 
or  a  requirement  that  all  proposed  services  of  a  particular 
practitioner  be  reviewed  by  a  physician  advisor  ("full 
concurrent  review"); 
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(c)  MCE  recommendations  for  continuing  education;  and 

(d)  other  MCE  recommendations  for  structural  changes  such  as 
physical  changes  in  hospital  organization  or  revisions  of 
hospital  privilege  procedures. 

The  ease  with  which  the  appropriateness  of  utilization  can  be  increased 
is  a  function  of  the  effectiveness  of  these  corrective  mechanisms. 

In  addition,  there  are  sanctions  available  to  the  Secretary  which  can 
be  invoked  in  severe  instances.    It  is  also  important  to  measure  the 
incidence  and  to  assess  the  effectiveness  of  these  sanctions  as 
corrective  mechanisms. 

B.  Management/ Pol  icy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Provide  technical  assistance  with  respect  to  corrective 
mechanisms. 

2.  Recommend  alternative  mechanisms  or  recommend  that  emphasis  be 
placed  on  specific  mechanisms. 

3.  Recommend  legislative  or  administrative  changes  in  the  program. 

C.  Analysis 

Incidence  of  Concurrent  Review  Corrections 

Analyses  of  the  first  two  sub-issues  will  be  conducted  as  two  special 
studies  -  one  at  the  completion  of  calendar  year  1977  and  one  at  the 
completion  of  calendar  year  1979.    Each  study  will  involve  observations 
from  all  PSROs  which  have  had  at  least  two  years  experience  beyond  the 
planning  stage.    To  assure  confidentiality  of  the  findings,  denial 
rate  statistics  and  other  measures  of  the  incidence  of  corrective 
actions  will  be  computed  at  the  PSRO  level.    Reports  of  the  analysis 
will  not  identify  any  individual  PSRO. 

In  each  of  the  PSRO  areas  in  the  study  the  different  variations  of 
special  requirements  for  concurrent  review  will  be  enumerated.  (Examples 
are  preadmission  certification,  full  concurrent  review,  required 
consultations,  etc.)    Measures  of  both  the  availability  of  these 
mechanisms  and  the  incidence  of  their  use  will  be  determined  by  geo- 
graphic region  and  for  the  nation  as  a  whole.    (Table  10,  Item  1) 
Pre-PSRO  experience  with  review  organizations  suggests  that  the  use  of 
formal  concurrent  review  corrections  will  be  infrequent.    This  should  not 
invalidate  the  results  of  the  second  phase  of  the  study  since  the  methodology 
to  be  employed  is  sensitive  to  small  values. 
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Impact  of  Concurrent  Review  Corrections 

The  next  phase  of  the  study  will  be  to  identify  all  physicians  in  each 
PSRO  area  in  the  study  who  have  been  recorded  as  either  a  Medicube 
admitting  physician  or  attending  physician  for  at  least  ten  proposed 
hospitalizations  in  each  of  the  two  fiscal  years  immediately  preceding 
the  study.    Excluding  physicians  with  nine  or  fewer  Medicube  patients 
will  eliminate  from  the  study  those  physicians  with  only  a  minimal 
participation  in  the  Medicube  inpatient  programs  and  will  increase 
confidence  in  denial  rate  statistics  computed  for  physicians  included 
in  the  study. 

In  each  PSRO  area  a  four  percent  sample  will  be  drawn  from  all 
physicians  not  excluded  from  the  study.    In  addition,  any  other 
physician  whose  AC  denial  rate  or  extension  request  denial  rate  is 
greater  than  that  of  ninety-nine  percent  of  all  physicians  in  the  area 
will  be  added  to  the  study.    This  will  allow  a  specific  examination  of 
the  impact  of  concurrent  review  denials  on  those  physicians  whose 
denial  rates  appear  to  be  well  in  excess  of  their  peers. 

The  total  sample  should  be  somewhat  less  than  five  percent  of  all 
physicians  not  excluded  from  the  study.    The  following  information 
much  of  which  can  be  derived  from  information  in  physician  profiles  will 
be  collected  on  each  physician  in  the  sample: 

(a)  AC  denial  rate  in  each  year  -  computed  as  the  ratio  of  a 
physician's  AC  denial  to  his  proposed  admissions, 

(b)  number  of  Medicube  admissions  proposed  in  each  year, 

(c)  extension  request  denial  rate  in  each  year  -  computed  as  the 
ratio  of  a  physician's  extension  request  denials  to  the 
total  of  his  extension  requests, 

(d)  number  of  Medicube  extension  requests  in  each  year, 

(e)  special  review  requirements  in  each  year, 

(f)  continuing  education  requirements  in  each  year, 

(g)  a  variable  to  indicate  whether  the  physician's  AC  denial  rate 
is  above  the  ninety-ninth  percentile, 

(h)  a  variable  to  indicate  whether  the  physician's  extension 
request  denial  rate  is  above  the  ninety-ninth  percentile, 

(i)  PSRO  membership  status, 

(j)    the  PSRO  area-wide  average  AC  denial  rate  for  the  two  years, 
(k)    the  PSRO  area-wide  average  extension  request  denial  rate 

for  the  two  years, 
(1)    the  percentage  change  in  the  PSRO  area-wide  AC  denial  rate 

from  the  first  year  to  the  second, 
(m)    the  percentage  change  in  the  PSRO  area-wide  extension 

request  denial  rate  from  the  first  year  to  the  second,  and 
(n)    the  year  the  PSRO  first  was  granted  conditional  status. 
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All  of  the  observations  in  the  sample,  without  physician  or  PSRO  area 
identifiers,  will  be  collected  for  use  in  a  regression  analysis.  The 
change  in  AC  denial  rates  and  the  change  1n  extension  request  denial 
rates  (Table  10,  Item  2b)  will  be  used  as  dependent  variables  in 
weighted  least  squares  regression  equations.    (The  individual  physician's 
caseloads  will  be  used  in  determining  the  weights.)    Policy  variables 
included  in  the  analysis  are  the  dummy  variables  for  special  review  and 
continuing  education  experience.    Qualifying  variables  include  PSRO 
membership  status,  the  dummy  variable  for  denials  at  the  ninety  ninth 
percentile,  the  PSRO  denial  rate  and  the  annual  change  in  that  rate, 
and  the  PSRO  cohort. 

Continuing  Education 

The  analysis  of  the  effectiveness  of  recommended  continuing  education 
as  a  corrective  mechanism  will  be  conducted  after  at  least  thirty 
PSROs  have  had  two  or  more  years  of  experience  beyond  the  planning  phase. 
The  collected  MCE  study  Abstracts  and  Restudy  Abstracts  will  be  examined 
to  identify  those  MCEs  which  recommend  continuing  education  and  for 
which  a  restudy  has  been  completed.    (Table  10,  Item  3a)    A  large, 
random  sample  of  these  abstracts  stratified  by  region  will  be  used  to 
identify  MCEs  for  further  study. 

A  special  survey  will  be  conducted  to  examine  the  actual  MCE  study 
findings  and  restudy  findings.    The  survey  will  determine  whether  the 
findings  give  quantitative  evidence  of  corrected  performance.    (Table  10, 
Item  3b)    No  individual  physician  or  PSRO,  however,  will  be  identified. 

The  survey  will  focus  on  an  indepth  examination  of  those  MCEs  which  have 
lead  to  the  greatest  improvement  and  those  which  have  shown  little  or 
no  improvement.    Interviews  will  be  conducted  with  members  of  the  MCE 
study  groups  and  rel event  PSRO  or  hospital  personnel  to  identify  those 
factors  that  most  contribute  to  or  most  inhibit  effective  correction 
through  continuing  education.    The  survey  will  analyze  the  importance 
of  the  following  factors:    delegation  of  review  authority,  PSRO  linkage 
with  existing  continuing  education  programs,  status  of  the  MCE  study 
members,  status  of  the  physician  or  physicians  for  whom  continuing 
education  is  recommended,  and  degree  of  support  from  the  hospital 
administrator. 

Other  MCE  Recommendations 

A  similar  survey  of  MCE  recommendations  will  be  conducted  to  enumerate 
other  suggested  corrective  mechanisms,  such  as  changes  in  a  hospital's 
procedures  for  granting  and  maintaining  privileges.    (Table  10,  Item  4) 
A  random  sample  of  such  MCEs  will  be  selected,  and  a  member  from  each 
study  group  will  be  interviewed  to  ascertain  any  special  circumstances 
which  made  the  recommendations  seem  appropriate  and  to  identify  measures 
of  impact.    The  analyses  of  the  survey  results  will  be  primarily 
descriptive,  however,  a  more  indepth  analysis  may  be  conducted  when  there 
is  evidence  of  a  particularly  effective  system  for  corrections. 
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Sanctions 

Title  XI  of  the  Act  gives  the  Secretary  two  sanctions.    As  a  result  of 
a  report  from  a  PSRO  that  an  individual  provider  has  not  fulfilled  his 
obligations  under  the  Social  Security  Act,  the  Secretary  may  prohibit 
that  provider  from  participating  on  a  reimbursement  basis  in  the  health 
benefit  programs  under  the  Act  or  may  impose  a  fine  equal  in  monetary 
value  to  the  health  services  provided  which  were  not  medically  necessary. 

At  the  end  of  each  fiscal  year  a  report  will  be  prepared  documenting  the 
incidence  of  such  sanctions  in  terms  of  the  numbers  of  providers  or 
practitioners  sanctioned,  their  distribution  by  region  and  PSRO  area, 
the  lengths  of  time  of  prohibited  participation  and  the  dollar  volume 
of  any  fines  imposed.    (Table  10,  Item  5a) 

Every  third  year  a  special  survey  will  be  conducted  to  ascertain  the 
impact  of  the  sanctions.    The  survey  will  involve  an  examination  of 
changes  in  performance  of  sanctioned  providers  or  practitioners, 
including  changes  in  concurrent  review  denial  rates,  and  changes  in  the 
volume  of  Medicube  practice,  e.g.,  withdrawal  from  Medicube  practice. 

D.  Research 

The  PSRO  legislation  has  been  interpreted  as  requiring  review  of  all 
Medicube  cases  admitted  to  institutions.    An  amendment  to  the  law, 
to  reduce  program  costs  by  allowing  sampling,  might  be  recommended  if 
it  could  be  shown  that  sampling  would  not  reduce  program  effectiveness. 
Research  on  this  question  could  be  conducted  even  in  the  absence  of 
program  amendments.    For  example,  were  it  demonstrated  that  specific 
characteristics  could  be  used  to  identify  "high-risk"  providers,  one 
form  of  sampling  would  use  these  characteristics  as  a  broad  screening 
device  to  focus  review  activities.    The  regression  analysis  described 
under  Impact  of  Concurrent  Review  Corrections  would  allow  an  examination 
of  the  impact  of  selected  physician  characteristics  on  performance  with 
respect  to  concurrent  review.    These  characteristics  could  include  age, 
specialty,  board  certification  status,  education,  practice  setting,  etc. 
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Issue  #9:    ARE  CRITERIA  AND  STANDARDS  BEING  DEVELOPED  AND  REVIEWED 
SYSTEMATICALLY  AND  ON  A  TIMELY  BASIS  AND  ARE  REVISIONS 
IMPLEMENTED? 


A.  Sub-Issues 

1.  What  selection,  control,  and  review  processes  are  utilized  to 
ensure  the  timely  development  of  new  and  revision  of  established 
criteria  and  standards? 

2.  What  factors  are  utilized  to  develop  new  and  revise  established 
criteria  and  standards? 

Rationale: 

Monitoring  the  appropriateness  and  quality  of  medical  care  against 
established  criteria  and  standards  for  admission  certification, 
continued  stay  review,  and  review  of  patient  management  with  respect  to 
diagnostic  and  therapeutic  procedures,  is  a  substantive  part  of  the 
PSRO  program.    Since  medical  care  is  not  a  static  process,  the 
criteria  and  standards  also  have  to  be  continuously  monitored  and 
reviewed  to  determine  if  they  are  still  appropriate  in  light  of  changes 
in  basic  medical  care  knowledge  (e.g.,  technology).    This  evaluation 
issue  relates  to  the  management  of  the  process  which  develops  and 
revises  the  criteria  and  standards  that  are  subsequently  used  to  monitor 
the  quality  of  health  care.    Since  there  is  a  great  number  of  recognized 
diagnoses  and  therapeutic  procedures  for  which  criteria  and  standards 
can  be  established,  the  selection,  control,  and  review  for  developing 
new  or  revising  existing  criteria  and  standards  requires  a  systematic 
management  process. 

B.  Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Establish  priorities  to  develop  and  revise  criteria  and 
standards. 

2.  Provide  technical  assistance  to  assure  that  criteria  and 
standards  are  being  developed  for  the  most  appropriate 
diagnoses  and  problems. 
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C.  Analysis 

This  issue  analysis  will  be  based  on  a  qualitative  assessment  of  the 
existing  standards  and  criteria  development,  review,  and  revision 
processes  in  each  operational  and  conditional  PSRO.    To  describe  these 
processes,  the  following  data  will  be  collected  (Table  11,  Item  1): 

1.  Who  develops  or  reviews  criteria  and  standards? 

2.  What  factors  were/are  considered  in  determining  those 
diagnoses  and  problems  for  which  criteria  and  standards 
will  be  developed  or  revised,  (e.g.,  10  most  frequent 
procedures)? 

3.  What  mechanisms  were/are  used  to  develop  or  review  criteria 
and  standards,  (e.g.,  MCEs,  review  of  ongoing  medical 
abstract  data,  selection  of  hospital  committee)? 

4.  How  often  are  the  criteria  and  standards  reviewed? 

5.  Were  revisions  implemented  where  indicated? 

6.  How  long  have  the  processes  been  in  operation? 

In  assessing  each  PSRO,  the  following  factors  will  be  taken  into 
consideration: 

1.  the  existence  or  non-existence  of  a  formal  process, 

2.  the  maturity  of  the  PSRO, 

3.  the  percent  of  Medicube  program  admissions  covered  by  the 
developed  criteria  and  standards  (Table  11,  Item  2), 

4.  the  criteria  and  standards  that  have  been  developed  and 
promulgated  by  the  program, 

5.  the  length  of  time  the  processes  have  been  in  operation,  and 

6.  the  diagnostic  categories  covered. 

In  addition,  there  are  some  indicators  for  detecting  instances  where 
changes  in  criteria  and  standards  may  be  necessary,  but  have  not  been 
implemented.    For  example,  there  may  be  a  significant  difference 
between  a  norm  and  its  related  criteria  and  standards.    This  difference 
could  result  from  a  valid  change  in  clinical  practice  which  has  not 
been  reflected  by  changes  in  the  criteria  and  standards.    Such  a 
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difference  is  only  an  indicator  because  there  are  other  conditions 
that  could  bring  about  the  deviation.    To  determine  if  these  causes 
stem  from  the  failure  of  timely  modifications  of  criteria  and  stand- 
ards, follow-up  studies  will  be  initiated  where  such  differences  are 
found. 

It  should  be  noted  that  a  portion  of  the  assessment  of  the  proper 
development  of  criteria  and  standards  will  be  covered  under 
Formative  Issue  8.    Under  that  issue  the  amount  of  agreement  between 
criteria  and  standards  sets  developed  by  the  various  PSROs  will  be 
measured.    The  index  of  agreement,  if  it  is  quite  high,  can  also  be 
construed  to  be  a  measure  of  proper  and  systematic  criteria  and 
standards  development. 

It  is  anticipated  that  the  information  required  to  conduct  these 
analyses  will  be  provided  through  the  PSRO  MIS.    Since  the  analyses 
for  this  issue  are  not  extensive,  these  assessments  will  be  conducted 
for  each  PSRO  annually. 
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Issue  #13:    HOW  EFFECTIVELY  AND  EFFICIENTLY  ARE  THE  VARIOUS  PSROs 
ORGANIZED? 


A.  Sub-Issues 

How  effectively  and  efficiently  are  the  various  PSROs  organized 
in  terms  of  their: 

1.  relationships  with  other  organizations  (health  care  institu- 
tions, state  councils,  support  centers,  etc.), 

2.  staffing  (number,  job  classifications,  qualifications  of 
personnel ,  etc. ) , 

3.  delegation  of  functions  (e.g.,  review  authority).,  and 

4.  review  activity  reimbursement  mechanisms? 


Rationale: 

Specific  organizational  structures  for  local  PSROs  are  not  prescribed 
either  by  law  or  by  regulation.    The  PSROs  are  thus  allowed  flexibility 
in  developing  and  adjusting  to  local  conditions.    It  is  likely, 
however,  that  certain  organizational  elements  will  better  facilitate 
the  effective  and  efficient  assumption  of  PSRO  obligations  and  activities 
than  others.    Program  managers  at  the  national  level  will  want  to 
encourage  the  adoption,  where  appropriate,  of  those  organizational 
elements  which  most  contribute  to  the  achievement  of  PSRO  goals,  and, 
conversely,  discourage  the  adoption  of  those  that  are  detrimental. 
Identification  of  such  organizational  factors  is  thus  essential. 

B.     Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Recommend  to  PSRO  the  adoption  of  certain  organizational 

structures. 

2.  Provide  additional  technical  assistance  to  PSROs  for  their 
administrative  functions. 

3.  Develop  policies  regarding  the  appropriate  training  and  other 
qualifications  for  PSRO  personnel. 
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4.  Recommend  possible  changes  in  policy  with  respect  to  the 
Secretary's  overriding  of  the  delegation  of  review  authority. 

5.  Develop  policies  with  respect  to  reimbursement  of  hospitals 
for  expenses  incurred  in  PSRO  review  activities. 

6.  Revise  reasonable  expenses  formulae  used  for  establishing 
budget  levels  in  PSRO  contracts. 


C.  Analysis 

The  analysis  will  be  carried  out  as  two  separate  special  studies. 
The  first  study  will  be  based  on  PSROs  with  at  least  one  year's 
experience  in  the  conditional  phase.    The  second  study  will  be 
identical  to  the  first  except  that  it  will  be  based  on  PSROs  which  have 
had  at  least  one  year  of  operational  status.    The  number  of  PSROs 
sampled  for  each  study  will  be  large  enough  to  provide  a  representative 
sample,  yet  small  enough  to  be  manageable.    The  studies  will  involve 
comparisons  between  different  PSRO  organizational  models  as  represented 
by  a  sample  of  PSROs  classified  according  to  the  following  characteristics 
(Table  12,  Item  1): 

1.  stage  of  development  (conditional,  operational) 

2.  the  number  of  Medicube  admissions  and  hospitals 

3.  organizational  arrangements  with  State  Councils 

4.  organizational  arrangements  with  support  centers 

5.  staffing  patterns,  in  terms  of  the  type,  number,  and 
qualifications  of  personnel 

6.  delegation  of  review  functions,  in  terms  of  the  proportion 
of  hospitals  delegated  concurrent  review,  the  proportion  of 
discharges  from  those  hospitals,  and/or  the  proportion  of  the 
budget  for  concurrent  review  incurred  in  hospitals  delegated 
review 

7.  review  activity  reimbursement,  in  terms  of  the  proportion  of 
total. PSRO  costs  reimbursed  through  direct  payments  to  the 
PSRO  (as  opposed  to  direct  payments  to  hospitals  for  that 
part  of  their  reasonable  expenses  incurred  through  PSRO 
review  activities) . 

The  different  PSRO  organizational  models  will  be  identified  through 
an  examination  of  reports  from  the  PSRO  Management  Information  System 
(PMIS)  and  Contract  Management  System. 
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Those  PSROs  selected  for  the  sample  will  be  surveyed  on  site  and 
examined  in  more  detail  to  provide  additional  data  for  the  organiza- 
tional assessment.    Comparisons  will  be  made  on  the  timeliness  of 
the  implementation  of  PSRO  functions  (and  adherence  to  scheduled 
implementation),  efficiency  of  the  review  processes  established  by  the 
PSRO,  and  the  costs  of  review.    Most  of  the  analyses  will  be  in  the 
form  of  narrative  reports  based  on  interviews  with  PSRO  staff  and 
informed  sources  in  the  local  areas.    The  analyses  for  each  study 
will  focus  on  seven  specific  topics  (Table  12,  Items  2-4): 

1.     the  impact  of  support  centers  on  the  development  of  PSROs 


2.  the  relationship  between  the  qualifications  and  organizational 
structure  of  the  executive  staff  and  the  timeliness  of 
implementation  of  PSRO  functions 

3.  the  impact  on  the  effectiveness  of  the  PSRO  review  component 
of  employing  personnel  who  have  participated  in  Federally 
sponsored  PSRO  training  programs 

4.  the  effect  of  interactions  between  PSRO  and  local  medical 
societies  and  institutions  on  the  development  of  that  PSRO 
and  the  effectiveness  of  its  review  program 

5.  the  impact  of  delegations  of  review  authority  on  the  over- 
all effectiveness  of  PSRO  reviews  (data  from  the  health 
care  review  issue  analyses  will  be  used  in  this  assessment) 

6.  the  impact  of  alternative  reimbursement  policies  on  the 
incentive  to  delegate  review  authority  and  the  incentive  to 
request  a  delegation  of  review  authority 

7.  the  effect  of  different  organizational  structures  on  the 
PSRO  budget  in  terms  of  total  cost,  cost  per  admission,  and 
cost  per  review. 
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SUMMATIVE  ISSUES 


Issue  #1:    ARE  ALL  MEDICUBE  BENEFICIARIES  COVERED  BY  ACTIVITIES  OF  PSROS? 

A.  Sub-issues 

1.  Have  PSRO  activities  been  implemented  in  all  hospitals  which 
participate  in  Medicube  benefit  programs  and  which  are  located 
in  PSRO  areas  with  conditional  or  operational  PSROs? 

2.  Are  all  Medicube  hospital  episodes  receiving  PSRO  review? 
Rationale 

The  intent  of  the  legislation  is  that  all  Medicube  beneficiaries  should 
be  covered  by  the  medical  review  activities  of  a  PSRO.    Thus,  it  is 
important  that  PSRO  activities  be  implemented  in  all  participating 
hospitals,  and  that  in  each  hospital  the  health  care  services  provided 
to  all  Medicube  beneficiaries  be  subject  to  review.    In  addition, 
blanketing  the  country  with  organizations  capable  of  assuring  that 
health  care  services  are  medically  necessary,  meet  professionally 
recognized  standards,  and  are  delivered  in  the  most  appropriate  setting 
will  provide  the  infrastructure  for  a  national  system  of  quality  assurance 
essential  to  the  implementation  of  a  National  Health  Insurance  program. 

B.  Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Recommend  changes  in  the  manner  in  which  PSRO  activities  are 
extended  throughout  the  hospitals  in  a  PSRO  area. 

2.  Determine  whether  the  legislative  requirement  has  been  met. 

C.  Analysis 

The  analysis  will  be  conducted  annually  in  all  areas  with  conditional  or 
operational  PSROs.    PMIS  and  PSRO  Contract  Reports  will  be  used  to 
identify  which  hospitals  in  each  PSRO  area  have  implemented  PSRO  activities 
either  on  a  delegated  or  non-delegated  basis.    Two  percentage  measures 
of  PSRO  hospital  coverage  will  be  computed.    The  first  will  be  the  simple 
measure  of  the  number  of  hospitals  participating  in  Medicube  with 
PSRO  activities  as  a  percentage  of  all  Medicube  participating  hospitals. 
(Table  13,  Item  la)    The  second  measure  will  use  the  annual  number  of 
Medicube  discharges  from  each  hospital  as  weights  in  a  weighted  percentage 
measure  of  coverage. 
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A  complementary  analysis  will  be  performed  annually  using  the  collected 
Medicube  PHDDS  records  for  each  PSRO  area,    PSRO  coverage  in  each  PSRO 
area  will  be  computed  as  the  ratio  of  the  annual  number  of  Medicube 
discharges  with  PSRO  review  to  the  annual  number  of  all  Medicube 
discharges.    (Table  13,  Item  2)    PSRO  coverage  in  the  nation  will  be 
computed  as  the  ratio  of  the  annual  number  of  Medicube  discharges  with 
PSRO  review  summed  over  all  active  PSRO  areas  (by  classification  of 
PSRO)  to  the  annual  number  of  Medicube  discharges  summed  over  all  PSRO 
areas  in  the  country.    (Table  13,  Item  3) 
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Issue  #2:    ARE  THE  HEALTH  CARE  REVIEW  ACTIVITIES  (AC,  CSR,  CQA,  MCE, 

AND  PROFILING)  AS  CARRIED  OUT  IN  THE  PSRO  PROGRAM,  ASSURING 
THE  QUALITY  AND  APPROPRIATENESS  OF  THE  UTILIZATION  OF  HEALTH 
SERVICES? 

A.  Sub-issues 
None 

B.  Management/Policy  Decision 

The  analysis  of  this  issue  will  provide  essential  information  for 
recommending  legislative  or  administrative  changes, 

C.  Analysis 

The  opening  section  of  Title  IX. B.  of  the  Social  Security  Act  contains 
these  words,  ".  .  ,  1t  is  the  purpose  of  this  part  to  assure,  through 
the  application  of  suitable  procedures  of  professional  standards  review, 
that  the  services  for  which  payment  may  be  made  under  the  Social 
Security  Act  will  conform  to  appropriate  professional  standards  for  the 
provision  of  health  care  and  that  payment  for  such  services  will  be  made 
only  when,  and  to  the  extent,  medically  necessary  ..."   Although  this 
issue  cuts  across  many  of  the  other  evaluation  issues  there  can  be 
no  doubt  that  the  procedures  of  professional  standards  review  must  be 
evaluated  with  respect  to  these  goals. 

The  analysis  of  this  issue  will  consist  primarily  of  a  synthesis  of  the 
results  of  other  evaluation  issues  and  other  evidence  available  from  the 
program  and  its  components.    The  procedures  of  professional  standards 
review  are  addressed  in  Formative  Issues  1,  2,  and  3.    The  national 
program  structure  is  addressed  in  Interim  Issue  1,  and  the  program 
coverage  of  the  Medicube  populations  is  addressed  in  Summative  Issue  1. 
The  synthesis  will  be  in  large  part  judgmental  in  an  effort  to  meld 
both  quantitative  and  descriptive  analyses  into  an  adequate  representation 
of  the  impact  of  the  program  as  a  whole.    The  synthesis  will  also  cover 
evidence  generated  external  to  the  national  program  evaluation.  This 
would  include  independent  research  conducted  by  local  PSROs  and  other 
research  which  demonstrated  PSRO  impact  such  as  might  be  conducted  under 
the  auspices  of  the  National  Center  for  Health  Statistics,  the  National 
Center  for  Health  Services  Research,  or  other  non-Federal  health  research 
groups. 

This  approach  necessitates  a  continuing  evaluation  of  this  issue.  Annual 
summaries  will  be  prepared  for  inclusion  in  the  annual  PSRO  report.  These 
summaries  will  be  able  to  cover  only  the  descriptive  analyses  in  the 
first  years  of  the  program,  but  will  include  some  more  quantitative 
analyses  with  the  report  for  fiscal  year  1977. 
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Issue  #3:    WHAT  HAS  BEEN  THE  IMPACT  OF  PSRO  ACTIVITIES  ON  PHYSICIAN 
BEHAVIOR? 


A.  Sub-Issues 

1.  How  have  physicians'  patterns  of  practice  changed  as  measured 
by  deviations  from  criteria  and  standards? 

2.  Has  physician  participation  in  the  Medicube  program  changed 
significantly? 

3.  What  impact  have  organizational  and  environmental  factors  had 
on  changes  in  physician  behavior? 

Rationale: 

The  PSRO  program  can  affect  the  behavior  of  physicians  primarily  in 
two  ways:    (1)  alter  physicians'  practices  with  respect  to  utilization 
and  quality  of  services,  and  (2)  alter  practitioners'  participation 
in  the  Medicube  programs. 

Formative  issues  1,  2,  3,  and  4  contain  methodologies  for  measuring 
the  impact  of  admission  certification,  continued  stay  review,  concurrent 
quality  assessment,  retrospective  review  and  corrective  mechanisms  on 
the  practice  of  medical  care.    Since  each  of  these  activities  can  have 
an  impact  on  physician  behavior,  it  is  necessary  to  pull  each  of  these 
issues  together  to  assess  the  degree  to  which  change  of  behavior  has 
occurred. 

With  respect  to  the  second  change,  it  has  been  alleged  that  the 
program  will  either  result  in  encumbrances  to  physicians'  practices 
or  be  perceived  as  presenting  such  encumbrances.    These  allegations 
lead  to  an  expectation  that  some  changes  in  physician  behavior  would 
be  reflected  in  their  Medicube  program  participation.    Thus,  the  analyis 
of  this  issue  should  also  be  directed  at  demonstrating  that  the 
anticipated  dropouts  are  not  occurring. 

Behavioral  change  is  a  complex  phenomenon,  however,  and  there  is  no 
reason  to  believe  that  physician  behavior  will  necessarily  change 
uniformly  across  the  program.    Also  when  physician  behavior  does  not 
change  uniformly  across  the  program,  it  may  be  due  to  other  than 
PSRO  program  effects.    For  example,  States  have  the  option  of  modify- 
ing benefits  in  their  Medicaid  program.    The  exercise  of  these  options 
can  also  have  the  effect  of  modifying  physician  behavior  .  Therefore 
to  gain  an  understanding  of  where  physician  behavior  has  changed  and 
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and  where  it  has  not,  and  whether  that  change  is  due  to  PSRO  program 
effects,  it  is  necessary  to  relate  that  behavior  to  those  organiza- 
tional and/or  environmental  factors  that  might  have  fostered  the 
change. 

The  factors  that  can  be  anticipated  to  influence  such  change  are: 

(a)  payment  setting  (fee  for  service  vs  prepaid  group  practice); 

(b)  independence  of  review  (delegated  vs  non-delegated  institutions; 

(c)  sufficiency  of  medical  personnel  (as  measured  by  physician/popula- 
tion ratios);  (d)  staff  privilege  patterns;  (e)  number  of  hospitals 

in  PSRO  areas;  and  (f)  geopolitical  characteristics  of  PSRO  area. 

B.  Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Recommend    administrative  and/or  legislative  changes  in  the 
program, 

2.  Provide  additional  technical  assistance  to  PSROs. 

C.  Analysis 

1.     Description  of  analyses 

Changes  in  Practice  Patterns 

This  analysis  will  consist  primarily  of  a  judgmental  synthesis  of  the 
results  of  portions  of  other  evaluation  issues  and  other  evidence 
available  from  sources  external  to  the  program  (Table  14,  Item  1). 
Formative  Issue  1  will  have  given  some  measurement  of  admission  and 
extension  denials.    Formative  issue  2  will  have  provided  some  indication 
of  the  degree  to  which  physician  behavior  will  have  improved  with 
respect  to  patient  outcome  and  deviations  from  criteria  and  standards. 
Similarly,  Formative  Issue  3  will  have  resulted  in  a  series  of  various 
improvements  as  a  result  of  changes  brought  about  by  suggestions 
resulting  from  MCE  studies,  while  Formative  Issue  4  will  have  provided 
measures  of  improvements  in  physician  performance  resulting  from 
corrective  mechanisms.    The  analysis  of  this  issue  would  also  include 
independent  research  conducted  by  local  PSROs  and  research  carried 
out  by  other  federal  health  research  groups,  such  as  the  Bureau  of 
Health  Services  Research  and  the  National  Center  for  Health  Statistics. 

Program  Participation 

This  analysis  will  be  performed  annually  on  a  national  and  regional 
basis  using  intermediary  and  carrier  records  as  the  data  source. 
Baseline  data  will  be  collected  for  the  year  1975. 
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There  are  two  ways  in  which  program  participation  may  be  affected  by 
PSROs.    One  would  be  reflected  in  a  change  in  total  number  of 
participating  practitioners  and  the  other  would  be  indicated  by  a  change 
in  the  degree  of  their  participation. 

To  measure  the  change  in  the  degree  of  participation  a  distribution  of 
the  Medicube  admissions  per  physician  will  be  computed  for  the  baseline 
year  and  each  succeeding  year.    (Table  14,  Item  2)  Variation  between 
years  will  be  examined  for  significance. 

The  number  of  physicians  who  have  either  terminated  or  initiated 
Medicube  program  participation  will  be  computed  each  year  to  measure 
the  net  change  in  the  number  of  participants.    This  analysis  will  take 
into  account  the  national  estimates  of  the  change  in  number  of 
practicing  physicians  that  usually  occur  due  to  physicians  either 
leaving  or  starting  their  practices.    Where  changes  are  traceable  to 
specific  states  and  to  the  Medicaid  population  they  will  be  checked 
against  changes  in  those  states'  benefits.    (Table  14,  Item  3). 

Organizational  and  Environmental  Factors 

During  the  course  of  the  analysis  of  Formative  Issue  4,  a  four  percent 
sample  will  have  been  drawn  to  examine  changes  in  concurrent  review 
denial  rates.    At  the  time  of  that  data  collection,  supplemental 
information  will  be  collected  on  every  fourth  physician  in  the  sample. 
No  individual  identifications  will  be  collected,  however,  and  all 
information  collected  will  remain  confidential. 

The  supplemental  information  will  be: 

1.  participation  in  prepaid  group  practice  -  a  dummy  variable 
set  equal  to  one  if  fifty  percent  or  more  of  the  physicians 
Medicube  practice  is  prepaid; 

2.  practice  in  delegated  hospitals  -  a  variable  equal  to  the 
percentage  of  the  physician's  Medicube  inpatient  discharges 
that  were  treated  in  delegated  hospitals;  and 

3.  the  PSRO  area  physician/population  ratio,  and 

4.  the  number  of  hospitals  in  the  PSRO  area. 

In  order  to  test  the  impact  of  these  factors  on  physician  performance 
(in  terms  of  concurrent  review  denial  rates)  these  observations  will 
be  included  in  a  weighted  least  squares  regression  analysis  of 
changes  in  denial  rates.    (The  weights  to  be  used  will  be  based  on  the 
physician's  annual  Medicube  caseload.)    The  dependent  variables,  in 
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two  separate  regression  equations,  will  be  the  change  in  the  admission 
certification  denial  rate  and  the  change  in  the  extension  request  denial 
rate,  respectively.    The  supplemental  information  on  organizational  and 
environmental  factors  will  provide  the  independent  variables.  (Table 
14,  Item  4)    The  dummy  variables  for  PSRO-SMSA  status  and  single 
state  PSROs  and  the  delegated  hospital  practice  variable  will  be 
included  as  qualifying  variables.    "Literature"  variables  include  the 
prepaid  practice  dummy  variable,  the  number  of  hospitals  for  which  the 
physician  has  admitting  privileges,  and  the  physician/population  ratio. 

B.  Comment 

Medicube  programs  are  known  to  have  claims  based  on  forgeries, 
duplicate  billings,  overcharges  and  fraudulent  services.    It  is 
believed  that  one  of  the  reasons  that  these  practices  exist  is  that 
the  submitting  individual  practitioner  is  often  the  sole  source  of 
justification  for  a  claim.    The  PSRO  program  may  have  a  profound 
impact  on  these  practices.    Although  such  beneficial  changes  would  be 
side-effects  and  not  part  of  the  main  thrust  of  the  program,  they  should 
not  go  unnoticed. 

There  are  no  direct  measures  of  this  beneficial  side-effect.  Therefore, 
an  evaluation  of  this  impact,  if  undertaken,  would  have  to  be  achieved 
by  means  of  an  indirect  method.    For  example,  some  States  have  been 
keeping  records  of  those  Medicaid  claims  that  are  identified  as 
resulting  from  some  of  these  practices.    Changes  in  rates  with  which 
these  practices  are  identified  could  possibly  serve  as  a  dependent 
variable  for  measuring  the  effect  of  the  PSRO  program  on  such  practices. 
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Issue  #4:    WHAT  HAS  BEEN  THE  IMPACT  OF  PSRO  ACTIVITIES  ON  INSTITUTIONAL 
BEHAVIOR? 


A.  Sub-issues 

1.  Have  the  organizational  structures  (e.g.,  committees)  or 
operating  procedures  (e.g.,  medical  records  abstracting) 
of  institutions  changed  as  a  result  of  PSRO  activities? 

2.  Have  there  been  changes  in  the  physical  structures  or  lay- 
outs of  institutions  as  a  result  of  PSRO  activities? 

3.  Has  there  been  a  significant  change  1n  participation  by 
institutions  in  the  Medicube  programs? 

4.  Do  hospital  occupancy  rates  Indicate  that  changes  In 
Medicube  utilization  induced  by  PSRO  review  have  been  off- 
set by  changes  in  utilization  by  non-Medicube  populations? 

Rationale: 

PSRO  review  of  the  quality  and  appropriateness  of  Medicube 
utilization  in  institutions  is  likely  to  have  an  effect  on  the 
organization  and  structure  of  those  institutions.  Measurement 
of  the  overall  impact  of  PSRO  will  have  to  Include  these  effects. 
In  addition,  within  the  institutions  there  may  be  significant  spillover 
effects  with  respect  to  the  non-Medicube  patients.    Indications  of 
these  effects  should  be  evident  in  the  composition  of  the  patient 
populations  served  by  the  institutions. 

B.  Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  information  for  the 
decisions  to: 

1.  Recommend  changes  in  program  regulations  or  in  the  legislation. 

2.  Provide  additional  technical  assistance  to  PSROs. 

C.  Analysis 

Changes  in  Organization  and  Structure 

At  the  completion  of  calendar  year  1976  a  special  study  will  be  conducted 
consisting  primarily  of  case  studies  of  a  sample  of  Institutions.  The 
sample  will  be  stratified  by  hospital  size,  population  density  of  the 
surrounding  community,  medical  school  affiliation,  and  delegated  review 
authority.    The  institutions  will  be  selected  from  PSRO  areas  which  have 
had  a  conditional  PSRO  for  at  least  eighteen  months. 
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Questional* res,  interviews,  and  on-site  observations  from  sampled 
institutions  will  be  used  to  obtain  the  required  information  for 
assessing  the  PSRO  impact  in  terms  of  changes  in  the  organization  or 
physical  structure.    The  Information  will  be  acquired  for  pre-  and 
post-  PSRO  periods.    The  pre-PSRO  period  data  will  be  obtained  immediately 
(before  the  end  of  calendar  year  75).    This  information  will  be  primarily 
a  description  of  the  organizational  structure  prior  to  PSRO,    After  18 
months,  site  visits  and/or  survey  will  be  conducted  to  obtain  a  descrip- 
tion of  the  current  (end  calendar  year  1976)  organizational  structure, 
thereby  identifying  organizational  changes  brought  about  by  the  direct 
or  indirect  influence  of  PSRO  activities.    MCEs  will  be  the  major  source 
for  obtaining  information  regarding  physicial  structure  changes. 

The  examination  of  changes  in  organizational  structure  will  focus  on 
policies,  functions,  and  activities  for: 

.  Admissions 

.  Utilization  Review  vis-a-vis  Concurrent  Review 

.  Medical  Records 

.  Accounting 

.  Hospital  Committees 

.  Clinical  hospital  departments 

.  Medical  discipline 

(Table  15,  Item  1) 

Impact  of  PSRO  activities  on  physical  changes  of  institutions  will  be 
assessed  with  regard  to: 

.    Physical  layout  (e.g.,  location  and  size/space  of  nursing 

stations,  or  clinical  departments) 
.    Changes  in  laboratory  and  technological  equipment 
.    Changes  in  size  of  facility  (e.g.,  increased  number  of  beds, 

square  footage) 

(Table  15,  Item  2) 

Changes  in  patient  populations  served 

A  special  study  will  be  conducted  annually  to  determine  the  extent  to 
which  institutions  in  PSRO  areas  are  significantly  increasing  or  de- 
creasing their  participation  in  the  Medicube  program.    The  study  will 
include  all  PSROs  which  have  been  in  conditional  or  operational  status 
for  at  least  one  year.    The  source  of  data  in  active  PSRO  areas  will 
be  PHDDS.    A  small,  matched  sample  of  PSRO  areas  without  PSROs  will  be 
used  for  comparison.    The  intermediaries  and  carriers  will  be  the 
source  of  information  for  the  number  of  Medicube  admissions  in  the 
non-active  PSRO  areas. 
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The  annual  numbers  of  Medicare  and  Medicaid  admissions  for  each  hospital 
in  the  PSRO  areas  under  study  will  be  tabulated  for  each  of  the  two 
most  recent  fiscal  years.    (Table  15,  Item  3)    All  hospitals  which 
exhibit  a  change  in  either  Medicare  or  Medicaid  admissions  in  excess 
of  ten  percent  will  be  identified  for  further  analysis. 

This  analysis  will  be  primarily  descriptive.    Those  hospitals  identified 
as  having  a  significant  change  in  Medicare  or  Medicaid  admissions  will 
be  examined  in  terms  of  the  following  factors:    a)  size,  b)  changes  in 
size,  c)  changes  in  the  availability  of  alternate  institutions,  d) 
requests  for  delegated  review  authority,  e)  changes  in  benefit  packages 
or  eligibility  requirements  in  their  state,  and  f)  any  specific  institu- 
tional responses  to  PSRO  review.    The  PSRO  areas  with  conditional  PSROs 
will  be  compared  to  the  areas  without  PSROs  in  terms  of  the  incidence 
and  importance  of  these  factors  in  order  to  distinguish  PSRO  induced 
effects  from  otherwise  autonomous  trends. 

Spillover  effects  on  occupancy  rates 

One  possible  response  to  decreases  in  Medicube  utilization  may  be 
observed  increases  in  non-Medicube  utilization.    A  special  study  will 
be  conducted  to  determine  the  extent  to  which  this  phenomenon  occurs, 
controlling  for  other  factors  which  are  also  likely  to  have  an  impact 
on  non-Medicube  utilization.    All  active  PSRO  areas  will  be  included 
in  the  study. 

The  change  in  non-Medicube  days  of  care  in  a  PSRO  will  be  used  as  the 
dependent  variable  in  a  regression  analysis.    The  percentage  of  Medicube 
patients  treated  in  delegated  hospitals  will  be  included  as  a 
policy  variable ,  and  the  change  in  Medicube  days  of  care  will  be  included 
as  a  qualifying  variable.    "Literature"  variables  to  be  included  in  the 
analysis  are  the  area-wide  occupancy  rate  and  the  change  in  the  area's 
total  population. 
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Issue  #5:    WHAT  HAS  BEEN  THE  EFFECT  OF  PSRO  ACTIVITIES  ON  HEALTH  CARE 
EXPENDITURES  IN  THE  UNITED  STATES? 


A.  Sub-issues 

1.  What  have  been  the  changes  in  health  care  expenditures  in  the 
United  States  with  respect  to: 

a.  total  health  care  expenditures, 

b.  total  Medicube  expenditures, 

c.  PSRO  program  expenditures? 

2.  What  has  been  the  effect  of  PSRO  activities  on  per  capita 
Medicube  expenditures? 

3.  What  has  been  the  change  in  hospital  expenditures  on  utilization 
review  (UR)  and  other  quality  assurance  activities  due  to  the 
introduction  of  PSROs? 

Rationale: 

Part  of  the  impetus  for  the  passage  of  the  PSRO  legislation  was  concern 
over  the  increase  in  expenditures  for  Medicare  and  Medicaid.    It  was 
believed  that  the  activities  of  the  PSROs  would  tend  to  moderate  any 
future  increases.    Changes  in  Medicube  expenditures  are  therefore  a 
significant  dimension  of  PSRO  impact.    Along  that  same  dimension  it  will 
be  important  to  measure  spillover  effects  on  total  health  care  expendi- 
tures where  such  measurement  is  possible. 

In  addition,  while  the  total  cost  of  PSROs  is  an  important  measure  of 
the  overall  effort  devoted  to  that  program,  it  is  important  to  note 
that  the  program  will  replace  current  federally  mandated  efforts  in  UR 
and  quality  assurance.    A  measure  of  the  additional  effort  incurred 
through  the  introduction  of  PSROs  will  be  the  difference  between  PSRO 
program  costs  and  the  current  expenditures  for  UR  and  quality  assurance. 

B.  Management/Policy  Decisions 

The  analyses  of  this  issue  will  provide  essential  information  for  the 
decisions  to  recommend  amendments  to  the  PSRO  legislation. 

C.  Analysis 

1.    Description  of  the  analyses 
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Changes  in  Health  Care  Expenditures 

At  the  end  of  each  calender  year  a  report  will  be  prepared  detailing 
the  level  of  health  care  expenditures  in  the  United  States.  The 
report  will  include  both  sources  and  distribution  of  funds,  and  will 
document  total  Medicube  expenditures  by  Program  and  the  total  PSRO 
budget  as  well  as  changes  in  those  expenditure  levels  from  year  to  year. 
(Table  16,  Item  1)    Sources  of  data  will  be  primarily  the  Office  of 
Research  and  Statistics  within  SSA,  the  Medical  Services  Administration 
in  SRS,  and  BQA. 

Changes  in  Per  Capita  Medicube  Expenditures 

A  study  involving  a  matched  sample  of  PSRO  areas  will  be  conducted 
biannual ly  to  assess  the  impact  of  PSRO  activities  on  changes  in 
Medicube  expenditures.    Areas  will  be  selected  on  the  basis  of  region, 
population  density,  Medicube  volume,  and  whether  they  are  in  single  or 
multiple  PSRO  states.    The  areas  will  be  chosen  so  that  half  of  the 
sample  will  have  conditional  or  operational  PSROs  and  half  will  have  a 
planning  PSRO  or  no  PSRO.    In  addition,  the  PSRO  areas  with  conditional 
or  operational  PSROs  will  be  chosen  to  assure  representativeness  of  those 
organizations  with  respect  to  their  dates  of  entry  Into  the  national 
program.    This  choice  will  enable  an  examination  of  any  changes  over  time 
in  the  impact  of  PSROs  on  health  expenditures. 

The  study  will  include  analyses  of  changes  in  Medicare  expenditures 
for  a  set  of  selected  diagnoses  in  addition  to  an  analysis  of  changes  in 
expenditures  for  all  diagnoses.    Those  diagnoses  chosen  will  be  of 
relatively  high  frequency  and  vary  as  much  as  possible  in  terms  of  per 
episode  expenditures.    As  in  Formative  Issue  #1,  the  lack  of  adequate 
national  data  on  individuals'  non-hospital  health  services  utilization 
dictates  primary  focus  on  inpatient  care.    As  a  result,  the  diagnoses 
chosen  for  these  analyses  will  be  those  for  which  care  is  provided 
primarily  in  hospitals  with  little  possibility  for  inpatient 
substitution.    Special  studies  on  spillover  impacts  on  ambulatory  or 
extended  care  expenditures,  however,  may  be  conducted  where  there 
is  adequate  local  data  (such  as  may  be  available  through  the  Medicaid 
Management  Information  Systems). 

In  each  area  the  following  statistics  will  be  collected: 

For  Medicaid  -  total  inpatient  hospital  expenditures 
number  of  inpatient  hospital  discharges 

For  Medicare  -  total  charges  (Part  A) 

total  intensive  care  charges 

total  operating  room  charges 

total  pharmacy  charges 

total  laboratory  charges 
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total  radiology  charges 

total  supply  charges 

total  anesthesia  charges 

total  inhalation  therapy  charges 

total  blood  charges 

total  inpatient  hospital  discharges 

total  charges,  Parts  A  and  B 

total  eligibles 

For  Medicare,  per  capita  hospital  expenditures  will  be  estimated  by 
dividing  total  charges  for  Part  A  by  the  number  of  eligibles.  A 
measure  of  PSRO  impact  on  expenditures  will  be  estimated  as  the 
difference  between  areas  with  PSROs  and  those  without  PSROs  in  terms 
of  the  percentage  changes  over  time  in  per  capita  hospital  expenditures. 

A  similar  analysis  will  be  conducted  using  data  from  the  Medicare  and 
Medicaid  programs.    Per  episode  expenditures  will  be  estimated  by 
dividing  total  hospital  expenditures  by  the  number  of  hospital  discharges. 
A  second  impact  measure  will  be  the  difference  in  percentage  changes 
over  time  in  per  episode  expenditures.    In  addition,  the  disaggregated 
Medicare  data  by  revenue  center  (intensive  care,  pharmacy,  radiology, 
etc.)  will  be  used  to  identify  any  particular  sources  of  observed 
differences  in  changes  in  per  episode  expenditures  over  time. 

Finally,  the  disaggregated  Medicare  hospital  data  will  be  supplemented 
by  Part  B  information  on  those  beneficiaries  who  have  been  recorded 
with  one  or  more  of  the  selected  diagnoses.    These  data  will  be 
analyzed  to  estimate  both  per  episode  and  per  eligible  expenditures  for 
each  of  the  diagnoses.    The  methodology  from  the  aforementioned  analyses 
will  be  replicated  to  identify  differences  and  sources  of  differences  in 
percentage  changes  in  per  eligible  expenditures.    (Table  16,  Item  2) 

Changes  in  Expenditures  for  Utilization  Review 

A  study  will  be  conducted  to  ascertain  pre-PSRO  expenditures  for 
utilization  review.    The  study  will  be  conducted  in  two  phases.  The 
first  will  focus  on  an  estimation  of  UR  expenditures  prior  to 
February  1975;  the  second  will  focus  on  UR  expenditures  in  compliance 
with  the  Medicare  and  Medicaid  UR  modifications  announced  in 
November  1974. 

Each  study's  estimates  will  be  based  on  a  sample  of  approximately 
forty  short-term  non-federal  hospitals  chosen  to  assure  representativeness 
by  region,  ownership,  hospital  size,  and  medical  school  affiliation. 
Each  hospital  in  the  samples  will  be  surveyed  to  ascertain  annual  UR 
expenditures  in  terms  of  personnel  costs  and  associated  overhead 
expenses  (including  paperwork  generated  to  document  the  UR  function). 
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The  first  phase  of  the  study  will  be  based  on  hospital  records  (or 
reconstructed  records)  of  UR  expenditures  in  calendar  1974,    The  second 
phase  will  be  based  on  hospital  records  for  the  period  of  calender 
1975,  and  the  hospitals  surveyed  shall  be  restricted  to  those  areas  in 
which  there  has  not  been  designated  a  conditional  PSRO. 

Both  phases  will  include  specific  consideration  of  the  relation  between 
UR  costs  and  coverage  under  UR  of  patient  groups  which  are  not 
Medicube  beneficiaries,    UR  costs  will  be  estimated  as  the  annual 
expenditures  for  UR  divided  by  the  number  of  discharges  subject  to 
utilization  review. 

The  impact  of  the  UR  modifications  will  be  measured  by  the  difference 
in  the  estimates  of  UR  costs  per  admission  from  the  two  phases  of  the 
study.    The  impact  of  PSRO  will  be  measured  by  the  difference  between 
the  second  phase  estimate  of  per  admission  UR  expense  and  the  analogous 
average  PSRO  expense.    (Table  16,  Item  3) 

2.  Comments 

a.  States  which  initiate  changes  in  the  Medicaid  benefit 
package  will  be  excluded  from  the  analysis  of  changes  in 
Medicube  expenditures  unless  each  part  of  the  sample  has 
a  PSRO  representative  from  such  a  state. 

b.  Current  Medicaid  data  collection  procedures  may  require 
that  the  information  needed  for  the  analyses  be  obtained 
from  State  Medicaid  Agencies  rather  than  from  SRS. 
Changes  in  these  procedures,  however,  may  enable 
disaggregated  analyses  to  be  performed  which  are  now 
possible  only  for  Medicare. 

c.  The  analyses  of  changes  in  Medicube  expenditures  will  not 
be  able  to  distinguish  modifications  due  to  changes  in 
patterns  of  practice  from  those  due  to  changes  in  prices. 
It  is  for  this  reason  that  the  analyses  focus  on  percentage 
changes  in  expenditures  rather  than  the  actual  expenditure 
levels  themselves,  and  it  is  assumed  that  PSRO  impact  on 
medical  care  prices  will  be  negligible.    A  study  that 
would  analyse  the  cost  impact  of  PSRO-induced  changes  in 
patterns  of  practice  is  suggested  in  the  research  section. 

D.  Research 

The  existence  of  the  PSRO  system  could  enable  a  study  that  would  examine 
changes  in  the  cost  of  selected  diagnoses.    Such  a  study  would  be  focused 
on  the  cost  impacts  of  changes  in  patterns  of  practice  and  would  provide 
a  side  benefit  in  a  cost  of  care  index  that  might  supplement  or  replace 
the  medical  care  component  of  the  Consumer  Price  Index. 
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The  study  would  be  conducted  using  data  from  selected  institutions  in 
a  sample  of  PSRO  areas  which  have  conditional  or  operational  PSROs, 
The  areas  would  be  stratified  with  respect  to  region  and  population 
density;  the  institutions  with  respect  to  ownership,  bed-size,  teaching 
affiliation,  and  review  delegation.    Data  collection  for  the  study  would 
cover  a  period  of  twenty-four  months.    The  first  twelve  would  constitute 
a  baseline  period,  the  latter  twelve,  a  comparison  period. 

The  study  would  be  based  on  a  subset  of  diagnoses  and/or  problems,  with 
relatively  high  incidence  and  varying  costs,  for  which  criteria  and 
standards  had  been  developed  in  the  study  areas.    Diseases  treated 
entirely  on  an  ambulatory  basis  would  be  excluded  from  consideration 
due  to  the  sheer  difficulty  of  collecting  selected  censuses  of  ambulatory 
encounters  as  well  as  the  particular  confidentiality  problems  that  would 
arise  in  collecting  such  data  on  non-Medicube  patients. 

Ambulatory  data  would  be  used,  however,  to  supplement  hospital  derived 
data.    Such  hospital  data  would  be  collected  on  all  discharges  from 
the  study  institutions  which  were  reported  with  a  principal  diagnosis 
(Contained  in  the  study  set.    This  information  would  include  (a)  an 
itemized  list  of  services  provided  during  the  hospital  stay,  (b)  the 
charges  for  those  services,  (c)  sources  of  payment,  (d)  indications 
of  whether  there  were  multiple  diagnoses,  and  (e)  whether  the 
treatment  was  in  compliance  with  the  applicable  criteria  and  standards. 
Ambulatory  data  would  include  both  utilization  and  itemized  charges. 

Separate  average  costs  of  treatment  would  be  computed  for  each  of  the 
diagnoses,  for  each  of  the  time  periods,  for  single  vs.  multiple 
diagnoses,  and  for  Medicube  and  non-Medicube  patients.    A  cost  index 
would  be  computed  as  the  ratio  of  the  weighted  sum  of  average  costs  of 
treatments  in  the  second  year  to  the  weighted  sum  of  what  the  average 
baseline  treatments  would  have  cost  at  the  second  year's  prices.  The 
weights  to  be  used  would  be  the  incidences  of  the  treatments  in  the 
second  year. 

PSRO  impact  on  changes  in  costs  should  occur  through  reductions  in 
the  number  of  cases  involving  inappropriate  utilization  and  reductions 
in  the  extent  of  inappropriate  utilization.    To  measure  this  impact  a 
second  set  of  computations  would  be  performed.    For  each  of  the  time 
periods,  in  each  of  the  diagnostic  categories,  and  for  each  of  the 
patient  populations,  two  additional  average  cost  statistics  would  be 
computed:    the  first  for  those  cases  in  compliance  with  the  criteria 
and  standards,  the  second  for  those  cases  not  in  compliance.  The 
weighted  sum  of  the  differences  between  the  overall  average  cost  of 
treatment  in  a  category  and  the  average  cost  for  a  compliant  case  in 
that  category,  as  a  proportion  of  the  weighted  sum  of  all  average 
costs  of  treatment  represents  that  fraction  of  average  expenditures 
for  health  care  incurred  through  utilization  not  in  substantial  compliance 
with  criteria  and  standards.    PSRO  impact  on  expenditures  would  be 
measured  by  changes  in  that  fraction  for  both  the  Medicube  and  non- 
Medicube  populations. 


00  >> 
C  CJ 
C 


T3 
U 

o 

CJ 
CU 


OS  X 


co 


< 


a. 


LU  LU 

_l  OS 

CD  ZD 

«=C  CO 

I —  < 


01 

u 

CO 

< 

< 

M 

OS 

00 

3 

CO 

pq 

oo 

C 

CO 

c 
c 


e 

it 

id 
u 

o 

60 

•H 

c 

O 

'— 

o 

U 

•H 

•H 

|Xc 

U 

00 

cu 

m 

PO 

a 

es 

s 

CO 

TJ  3 

C  4-1 

0)  -H 

o.  -a 

x  c 

QJ  QJ 

QJ  &  4J 

^  cu  m 

cfl  00 

CJ  CU  T> 
XI  3 

x:  3  xj 

•u  a 

i-H  i-l  O 

a  -a  pi 

cu  a)  oo 

Xi  S  IX 


o  o  o 


CO  X)  CJ 


XJ-  4J  4J 


CO 
3 
C 
C 
CO 
•H 

pq 


<  w 
co  OS 


101 


a 


i 


u 

CO 

QJ  i-l 
"3 
C 

C  O 
•H  O 

co  x: 

QJ  4J 
iH 
3  -S 


— 

B 
CU 

CU 


I 

l-i  -H 

CO  TT 

Cfl  CU  C 
iH    Ss  O 

CO  CJ 

C  C 

O  t-f  XI 
■H  4J 
4J    Cfl  iH 

cfl  qj  3 

i-i  00 
H 


H  C 

cfl 

U 

cfl  x: 

QJ  4-1 


x: 

CJ 

cfl 
cu 


X  cfl  cfl 


3  "H 


o  x: 

u 


I     03    Cfl  I 


O 
OS 

i-H  CO 

Cfl  X 
4-1 

o  c 


Q 
ss 
CO 
X 


(0 

a 
o 


cfl  i-l  CO 

C  CU 

O  CO  i-i 

H  CU  CO 

U  U 

CO  3  C 

^  u  a 

cu  th  co 

a  -a  x 

o  c 

cu  x; 

CO  QJ 

H  0) 

cfl  i-h  x: 

C  cfl 

o  w 

o  c 


CO 

cfl 

0)  CJ 

U  (H 

cfl  X 
E 

D  cfl 

^  CO 


EU 


u  i<  O 
cu  eS 
x  co 

O  X 


Vj  O 

cu 

XI  u-i 

E  H 

3  cfl 

C  Xi 


C  CO 
•H  t-H 
Cfl 

CN  C 
O 

i-i  iH 

CO  4-1 

0)  iH 
^  T3 
C 

C  O 

T-l  O 

CO  4-1 

0)  3 

U  0 

3  Xi 

4J  4-1 
■H  i-l 

C 

cu  co 

&s 

CU  Sj 
cfl 

i— I 

cfl  O 

4J  OS 

0  co 

4-1  X 


CO 

0) 

M 

3  CO 

4-1  CU 

cu 

•H  iJ 

Xi 

■a  3 

3 

C  4-1 

CJ 

CU  i-l 

•H 

X  T3 

■a 

X  C 

qj  gj 

a 

cu  X 

4-1 

CO 

i-i  cu 

CU 

4-1 

cfl 

60 

o  cu 

XI 

(X 

XJ 

3 

CO 

x:  3 

XI 

o 

4-1  cj 

n 

o 

I-l 

cfl  -a 

OS 

QJ 

co 

CU  QJ 

CO 

X 

x:  S 

X 

C 

3 

■H  iH 

1—1 

■H 

4-1 

CO  Cfl 

Cfl 

■H 

4-1  4J 

4-1 

QJ 

— 

o  o 

o 

60 

3 

4J  4-1 

4-1 

G 

cu 

cfl 

x: 

& 

CO  XI 

CJ 

c_> 

CN 

Cfl 
3 
C 


3 

CO 


a 
cu 
- 

ro 


T— 1 

CO 

cfl  pa 

< 

- 

- 

4-1 

CU 

E 

cu 

c 

•H  4-1 

4-1 

u 

O 

- 

1 

X  s- 

r. 

o 

4J 

> 

fl 

co  co 

Cfl 

CJ 

cu 

•rl 

O  X 

x 

ns 

c 

X! 

x: 

CU 

60  i- 

o 

— 

QJ 

CU 

> 

c  >>  o 

</) 

— 

X! 

■- 

4-1  U 

u 

•h  a  w 

c 

0) 

-J 

P3 

Cfl 

C  Cfl 

r. 

CO 

4-1    Cfl  Cfl 

— 

c^x: 

Efl 

CTn 

cfl 

CJ 

QJ  CJ 

c 

co  E  u 

a 

i-l  4-> 

— 

is 

iH 

•H 

■H  tH 

■H 

cu 

)-    U  0 

■H 

X  oo 

c: 

o 

1 

m 

4J  "0 

— 

4J 

CU    Cfl  X3 

X  OJ 

o 

QJ 

Cfl  QJ 

c 

X  JZ  Cfl 

cfl 

3  C 

: 

iH 

i-i 

CT 

x  s: 

2 

■H 

0     X  rH 

u 

co  ro 

■H 

X 

X 

— 

•a 

QJ 

> 

H 

O  T3 
>  C 
C  CO 
■H 

Cfl 
P 

c 


cu 

C  4-1  i-l 

O  Cfl  CU 

CO  CJ  X) 


l-i  -H 
CU  M-i 
X  -H 


u-i  cfl  ► 
O    3  CO 

cr  qj 
^      -h  x 
l 

XI  -H  Cfl 

E  QJ  i— l  i-i 
3  x:  co  c 
C  4-j  co  2 


CO 
QJ 

^1 

3 

4-1 

*H 

u 

& 

cu 


— 

> 

QJ 
s- 

C 

o 

•H  Cfl 

4-1  QJ 

Cfl  4J 

S!  Cfl 

■H  E 


O 


S- 

Cj 

QJ 

c 

<t- 

S- 

ro 

<*- 

!_ 

4P 

00 

C 

— 

— 

— 

T3 

c 

E 

QJ 

CM 

c 

D. 

— 

N 

X 

00 

c 

QJ 

QJ 

4p 

fO 

QJ 

QJ 

E  x: 

x: 

s- 

cz 

c 

U 

rz 

Hj- 

— 

S- 

□ 

c 

cu 

— 

c 

x: 

CO 

c 

- 

c 

■  r— 

00 

= 

ro 

+J 

c 

c 

ro 

- 

> 

o 

U 

QJ 

QJ 

LO 

— 

— 

CU 

!- 

!_ 

QJ 

QJ 

— 

a. 

ax: 

S- 

OJ 

+-> 

ro 

5- 

c 

c 

U 

S- 

ro 

CO 

c 

~- 

cu 

QJ 

00 

[_ 

~ 

u 

C 

ro 

01 

00 

ro 

C4U 

x: 

CO  xz 

— 

u 

C 

c 

o 

c 

It 

— 

c 

ro 

c 

c 

N 

c 

4P 

CO 

+-> 

— 1 

35  S_ 

—  o 

(B  Q. 

>  O 

QJ  S- 
D. 

QJ 


CO  CO 

E  E 
S-  i- 

o  x 

S3  O 
5- 

OJ  CX 

x:  a. 

I-  ro 


102 


SECONDARY  EFFECTS  ISSUE 


Issue  #6:    WHAT  HAS  BEEN  THE  IMPACT  OF  PSRO  ACTIVITIES  ON  HEALTH  STATUS? 


A.  Sub-issues 
None 

B.  Management/Policy  Decisions 

The  analysis  of  this  issue  will  provide  essential  Information  for 
recommending  legislative  and/or  administrative  change. 

C.  Analysis 

In  order  to  provide  a  more  comprehensive  assessment  of  the  impact  of 
PSRO  on  the  quality  of  care,  it  would  be  necessary  to  measure  the  changes 
in  the  health  status  of  the  population  affected.    However,  there  are  a 
number  of  serious  problems  in  using  health  status  as  a  measure  of  the 
impact  of  the  PSRO  program.    Some  of  these  problems  are: 

(a)  The  current  state-of-the-art  of  health  status  assessment 

is  primitive,  at  best.    Few  health  status  measures  have  been 
validated  sufficiently  to  be  considered  for  use  1n  evaluation. 

(b)  Health  status  is  not  independent  of  other  non-medical  factors, 
such  as  the  lifestyle,  education,  socio-economic  status, 

and  environmental  conditions  of  a  population. 

(c)  Health  status  may  be  affected  by  other  recently  introduced 
health  programs  such  as  Health  Maintenance  Organizations, 
Emergency  Medical  Services  and  Health  Systems  Agencies. 

Although  this  issue  is  very  Important,  the  completion  of  this  analysis 
must  await  further  development  of  health  status  measures.    This  activity 
would  most  appropriately  be  part  of  a  research  program. 


Chapter  V 
Implementation 
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CHAPTER  V 
IMPLEMENTATION 


The  evaluation  of  the  PSRO  program  will  be  implemented  during  the  next 
five  years  according  to  the  schedule  outline  in  Table  17.    Each  A 
priority  issue  and  its  sub-issues  have  been  included.    Due  in  large 
part  to  the  pattern  of  growth  of  the  program,  certain  analyses  cannot 
be  initiated  until  a  sufficient  number  of  mature  PSROs  have  conducted 
medical  care  review  for  at  least  one  year.    Table  17  also  reflects 
the  fact  that  many  of  the  issues  will  be  examined  several  times.  With 
respect  to  those  issues  whose  analyses  cannot  be  completed  for 
several  years,  a  preliminary  study  will  be  conducted  so  that  important 
evaluative  information  can  be  made  available  at  an  earlier  date. 
Other  issues  will  require  periodic  updates  to  facilitate  their  continu- 
ing analysis. 

An  estimate  of  the  funds  required  to  carry  out  the  evaluation  plan  are 
presented  in  Table  18.    This  budget  was  based  upon  the  following 
assumptions: 

1.  computer  services  will  be  obtained  from  the  operating 
agency,  thus,  avoiding  certain  costs  connected  with 
separate  contracting; 

2.  PHDDS  would  be  available  in  a  national  computerized  data 
base; 

3.  data  collected  by  Medicare/Medicaid  intermediaries  and 
carriers  would  be  available  to  the  evaluation  staff  at 
little  cost  over  existing  arrangements; 

4.  a  large  part  of  the  evaluation  would  be  performed  with  an 
in-house  staff  of  approximately  7-10  professionals.  With 
a  reduced  number  of  staff  the  cost  estimates  could  be 
increased  by  as  much  as  $300,000  per  year. 

5.  certain  data  not  presently  required  by  the  PMIS  Federal 
Reporting  will  be  collected  by  PSROs,  so  that  extensive 
special  data  collection  would  not  be  necessary.  The 
PSROs  would  be  reimbursed,  but  the  cost  would  be  less 
than  if  the  data  were  obtained  by  an  independent 
contractor. 

6.  FY  76  does  not  include  the  cost  of  collecting  the  required 
baseline  data  because  a  method  for  collecting  the  data 
(Appendix  B)  has  not  as  yet  been  agreed  upon. 
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In  the  event  the  projected  funding  levels  are  not  available,  it  may  be 
necessary  to  omit  the  implementation  of  some  of  the  "A"  priority  issue 
analyses.    It  is  difficult  to  advance  a  clear  rationale  as  to  the  order 
in  which  the  analyses  should  be  eliminated,  but  there  is  clearly  a  minimum 
set  of  issues  without  which  the  NPSRC  would  not  be  able  to  provide  to  the 
Secretary  and  the  Congress  adequate  information  regarding  the  "effective 
accomplishment  of  the  purposes  and  objectives"  of  the  PSRO  program.  This 
minimum  set  consists  of  Formative  issues  1,  3  and  4  and  Summative  issue  5 
as  these  are  the  issues  that  most  directly  deal  with  the  impact  of  the 
PSRO  program  on  cost,  quality  and  the  utilization  of  services  as  well 
as  the  specific  mechanisms  mandated  by  the  legislation  for  affecting 
those  factors.    The  total  cost  estimated  for  implementing  this  minimal 
set  is  $1.5  mil  1  ion. 

In  developing  the  implementation  plan  it  was  assumed  that  the  evaluation, 
except  for  those  issues  which  are  primarily  of  a  monitoring  nature, 
would  be  carried  out  by  an  organization  separate  from  the  agency  implement- 
ing the  program.    Such  an  organization  should  have  a  strong  analytical  and 
evaluative  capability  and  wide  responsibilities  in  most  aspects  of  health 
care,  particular  those  relating  to  quality  assurance.    The  reasons  for  this 
assumption  are: 

1.  There  is  a  tendency  for  the  operating  agency  to  focus  on 
short  range  issues  rather  than  on  long  term  program  effects. 

2.  The  role  of  the  operating  agency  is  naturally  one  of  program 
advocacy,  thus,  preventing  complete  objectivity. 

3.  Because  of  (2)  above,  there  is  a  serious  credibility  problem 
with  the  results  of  an  evaluation  conducted  by  an 
operating  agency. 

Responsibility  for  carrying  out  those  issues  consisting  primarily  of 
monitoring  various  parts  of  the  program  should  rest  with  the  operat- 
ing agency.    The  results  of  these  analysis  should  then  be  made  avail- 
able to  the  principal  evaluating  organization.    Table  19  contains 
the  recommended  issue  analyses  assignments. 
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TABLE  17 

ANALYSES/STUDIES  COMPLETION  SCHEDULE 


1976  1977  1978  1979 


Caleiidar/Quarters 

Tssiips /Snh  I«:iip<:  - — 

,  1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

1 

2 

3 

4 

Interim  1:    Proqram  Status 

X 

X 

X 

X 

Formative  1:    Concurrent  Review 

Sub  1 :          Extent  of  AC 

X 

2:          Extent  or  CSR 

v 

X 

_           Concurrent  Review 
3:  Effectiveness 

X 

X 

4:          Denial  Extension 

X 

Utilization  Change  on 
5:          Medicube  Expenditures 

X 

X 

6:          Net  Savinas 

X 

X 

Concurrent  Quality 
Formative  2:  Assessment 

Sub  1:          Extent  of  CQA1 

2:          Deviation  Reduction^ 

O.               LOSL  OT  LyH 

rormaxive         kg Lrospec i i ve  Keview 

oUD  I.           rroDiem  icieniiT  icaLion 

Y 
A 

V 
A 

V 

A 

c .            Mclnouo  luyiLa  i  ouuriuricbo 

Y 

A 

x 

j.               lil/L  LTTcCLlVcricbb 

Y 

A 

Y 

A 

x 

4-.                FILL    nUUc  I    LI  I  cl  U  Vcllcbo 

x 

Fnrmativp  4'     forrpftivp  Mechanisms 

Sub  1:          Use  of  Corrective  Mech. 

X 

?.          Impact  of  Concurrent 
Review  Denials 

X 

3:          Continuing  Education 

9 

X 

4:          MCE  Recommendations 

X 

5:  Sanctions 

L 

1 

1 

X 
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(Continued) 

TABLE  17 

ANALYSES/STUDIES  COMPLETION  SCHEDULE 


1976  1977  1978  ,  1979 


■  Calgjidar/Quarters 

R<;iiP<;/Siih  R«;iip<;                    "     "        —  . 

1 

2 

3 

4 

1 

2 

3 

4 

1 

I 

3 

4 

1 

2 

3 

4 

Development  and  Review 
Formative  9:    of  Criteria/Standards 

Sub  1:          Svstems  Used 

X 

X 

X 

2:          Factors  Impact 

X 

X 

X 

Formative  13:  Impact  of  Organization 

X 

X 

Summative  1:    Proqram  Coveraqe 

Sub  1:          All  hospitals 

X 

X 

X 

2:          All  Medicube  Admissions 

X 

X 

X 

Health  Care  Review  Assuri 
Summative  2:    Quality  X  Appropriating 

X 

X 

of  Services 

Clim5n„0  o .    PSRO  Impact  on 
Summat^e  3-   PhvsiciSn  Behavior 

Sub  1:          Practice  Patterns 

X 

2:          Proqram  Participation 

X 

X 

X 

X 

3:          Identification  of  factors 

X 

PSRO  Impact  on 
Summative  4:    Institutional  Behavior 

iuu  i.          urgam  zau  i  on  structure 

Y 

A  . 

Y 
A 

2:          Physical  Structure 

X 

Shift  in  population 
3:  served 

X 

X 

X 

X 

4:          Occupancy  rate  Change 

X 

1 

i 

i 

TABLE  17  (Continued) 
ANALYSES/STUDIES  COMPLETION  SCHEDULE 


1976 


1977 


1978 


1979 


PSRO  Impact  on  Health 
Summative  5:    c.arp  Fxpenrii tliTfiS 


Sub  L 


Change  in  Expendi tliEfiS 


2l 


Per  Capita  Change 


UR  Costs 


Baseline  Data  Collection 
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Notes: 

1.  Schedule  is  dependent  on  Private  Initiative  PSRO  Study. 

2.  Unknown  at  this  time  because  the  number  of  PSROs  that  will 
implement  CQA  is  not  known. 

3.  A  second  study  is  needed  to  verify  estimates  of  new  UR  regulation 

costs . 
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TABLE  18 
EVALUATION  PLAN  BUDGET 

Fiscal  Year 


Fiscal  Year 

1976 

1977 

1978 

1979 

Contracting  $' 

300, 0002 

1 ,300,000 

700,000 

700,000 

S&E  $] 

150,000 

300,000 

300,000 

300,000 

Total 

450,000 

1  ,600,000 

1  ,000,000 

1  ,000,000 

"'These  are  in  1975  dollars. 

2Does  not  include  cost  of  collecting  required  baseline  data  which 
may  be  subs  tan  ti  al . 
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TABLE  19 
Issue  Analysis  Assignment 


Issue 


Operating 
Agency 


Non-Operating 
Agency 


Interim  1:    Program  Status 

Formative  1:  Concurrent  Review 

Sub  1:  Extent  of  AC 

2:  Extent  of  CSR 

2:  Extended  Stays 

3:  Effectiveness 

4:  Denial  Extension 

5:  Utilization  Change  on 
Medicube  Expenditures 

6:  Net  Savings 

Formative  2:  Concurrent  Quality 
Assessment 

Sub  1:  Extent  of  CQA 

2:  Deviation  Reduction 

3:  Cost  of  CQA 

Formative  3:  Retrospective  Review 

Sub  1:  Problem  Identification 

2:  Methodological  adequacy 

3:  MCE  Effectiveness 

4:  MCE  Model  Effectiveness 

Formative  4:  Corrective  Mechanisms 

Sub  1:  Use  of  Corrective  Mech. 


m 


Issue 


Operati  ng 
Agency 


Non-Operating 
Agency 


2: 

3 
4 
5 

Formative  9: 

Sub  1: 
2: 

Formative  13: 
Summative  1: 
Sub  1: 
2: 

Summative  2: 

Summative  3: 
Sub  1 

2: 
3: 

Summative  4: 


Impact  of  Concurrent 
Review  Denials 

Continuing  Education 

MCE  Recommendations 

Sanctions 

Development  and  Review 
of  Criteria/Standards 

Systems  Used 

Factors  Impact 

Impact  of  Organization 

Program  Coverage 

All  hospitals 

All  Medicube  Admissions 

Health  Care  Review  Assuring 

Quality  &  Appropriateness  of  Services 

PSRO  Impact  on  Physician  Behavior 

Practice  Patterns 

Program  Participation 

Identification  of  factors 

PSRO  Impact  on  Institutional 
Behavior 


X 
X 


X 
X 


Sub  1: 


Organization  Structure 
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Operating  Non-Operating 
Issue  Agency  Agency 


2:  Physical    Structure  X 

3:  Shift  in  population  served  X 

4:  Occupancy  rate  Change  X 

Summative  5:  PSRO  Impact  on  Health  Care 
Expenditures 

Sub  1:  Change  in  Expenditures  X 

2:  Per  Capita  Change  X 

3:  UR  Costs  X 


Appendix  A 


Defining  Clinical  Diagnoses  and 
Procedures  for  Use  in  the  Evaluation 
of  the  Professional  Standards  Review 
Organization  Program 
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APPENDIX  A 

CLINICAL  DIAGNOSES  AND  PROCEDURES  FOR  USE  IN  THE  EVALUATION 
OF  THE  PROFESSIONAL  STANDARDS  REVIEW  ORGANIZATION  PROGRAM 


I.  Introduction 

The  National  Professional  Standards  Review  Council  has  developed  eleven 
program  goals  which  provide  direction  for  the  total  program  and  the 
criteria  against  which  the  program  can  be  evaluated.    These  goals  (See 
Chapter  II)  were  partitioned  into  four  basic  areas: 

1.  Goals  related  to  quality  of  services 

2.  Goals  related  to  resource  allocation 

3.  Goals  related  to  program  implementation  and  acceptance 

4.  Goals  related  to  professional  education. 

The  PSRO  Program  evaluation  plan  has  then  specified  several  high  priority 
issues  which  are  detailed  in  Chapter  III  of  this  document.    The  collection 
and  analysis  of  detailed  clinical  data  is  primarily  required  for  those 
issues  related  to  the  first  two  areas,  quality  of  service  and  resource 
allocation.    These  issues  are: 

1.  Have  admission  certification  and  continued  stay  review 
(concurrent  review)  been  effective  methods  of  reducing 
medically  unnecessary  hospital  utilization? 

2.  Is  Concurrent  Quality  Assessment  (CQA),  as  carried  out  in 
the  PSRO  program,  an  effective  method  of  assuring  the 
appropriate  utilization  of  services  (diagnostic  and 
therapeutic  procedures)? 

3.  Is  retrospective  health  care  review,  as  carried  out  in 
the  PSRO  program,  an  effective  method  of  assessing  the 
quality  and  appropriateness  of  the  utilization  of 
services? 

4.  How  effective  are  the  various  corrective  mechanisms  applied 
to  institutions  or  practitioners  for  correcting  inappropriate 
utilization  of  services  and/or  improving  the  quality  of  care? 

5.  How  appropriate  is  the  variation  among  PSRO's  in  terms  of 
standards  and  criteria  for  admission  certification,  continued 
stay  review,  and  review  of  patient  management  with  respect  to 
diagnostic  and  therapeutic  procedures? 
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6.  Are  the  health  care  review  activities  (AC,  CSR,  CQA,  MCE  and 
profiling),  as  carried  out  in  the  PSRO  program,  assuring  the 
quality  and  appropriateness  of  the  utilization  of  health 
services? 

7.  What  has  been  the  impact  of  PSRO  activities  on  physician 
behavior? 

8.  What  has  been  the  impact  of  PSRO  activities  on  institutional 
behavior? 

9.  What  has  been  the  impact  of  PSRO  activities  on  the  health 
status  of  the  Medicube  and  non-Medicube  populations? 

Because  of  the  costs  associated  with  collecting  and  analyzing  clinical 
data  for  the  evaluation,  it  Is  not  practical  or  feasible  to  consider 
all  possible  diagnostic  categories  and/or  procedures.    Therefore,  the 
purpose  of  this  portion  of  the  evaluation  plan  is  to  define  a  set  of 
clinical  diagnoses  and  procedures  which  will  be  particularly  useful 
for  measuring  the  quality,  cost  and  utilization  of  health  care  services 
and  for  which  data  will  be  collected  for  a  period  prior  to  the 
implementation  of  PSROs  (baseline)  and  on  a  regular  and/or  special 
study  basis. 

The  remainder  of  the  paper  is  divided  into  four  sections.    The  first 
contains  a  review  of  previous  methods  for  selecting  clinical  conditions 
in  quality  of  care  studies.    The  second  defines  selection  criteria  and 
describes  the  selection  process.    The  third  contains  the  recommended 
diagnoses  and  procedures.    Finally  a  discussion  of  the  technical 
problems  and  currently  available  data  sources  is  contained  1n  Tab  A. 


III.    Previous  Methods  for  Selecting  Clinical  Conditions  in  Quality  of 
Care  Studies 

While  there  are  no  previous  quality  of  care  studies  of  a  scope  and 
magnitude  equivalent  to  that  which  is  required  by  the  PSRO  program, 
several  methods  for  selecting  clinical  conditions  have  been  developed 
which  have  a  high  degree  of  applicability  to  the  PSRO  Program  Evaluation. 

Kessner  (1)  has  developed  detailed  criteria  which  were  utilized  to 
define  "tracer  diseases"  (see  Tab  B)  in  the  measurement  of  quality  of 
care.    These  criteria  are  as  follows: 

1.    A  tracer  should  have  a  definite  functional  impact 
(emphasis  added).    The  over-riding  purpose  of  the 
tracer  approach  is  to  focus  on  specific  conditions 
that  reflect  the  activities  of  health  professionals. 
Conditions  that  are  unlikely  to  be  treated  and  those 
that  cause  negligible  functional  impairment  are  not 
useful . 
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2.  A  tracer  should  be  relatively  well  defined  and  easy  to 
diagnose,    Dermatologic  conditions  have  a  clear  functional 
impact.    The  difficulties,  however,  of  defining  clear-cut 
pathologic  entities  lessen  their  utility  as  tracers.  In 
contrast,  it  is  relatively  easy  to  identify  a  population 
of  patients  with  a  hematocrit  below  a  specified  level  and 
further  to  diagnose  those  with  iron-deficiency  anemia. 

3.  Prevalence  rates  should  be  high  enough  to  permit  the 
collection  of  adequate  data  from  a  limited  population 
sample.    If  an  adequate  number  of  cases  cannot  be 
studied,  it  is  difficult  to  evaluate  even  the  most 
important  variables  in  relation  to  the  set  of  tracers. 

4.  The  natural  history  of  the  condition  should  vary  with 
utilization  and  effectiveness  of  medical  care.  Ideally, 
in  evaluation  of  a  delivery  system,  the  conditions  under 
study  should  be  sensitive  to  the  quality  or  quantity 

(or  both)  of  the  service  received  by  the  patient.    It  is 
inappropriate  to  use  conditions  for  which  health  services 
do  not  alter  the  progress  of  the  disease. 

5.  The  techniques  of  medical  management  of  the  condition 
should  be  well  defined  for  at  least  one  of  the  following 
processes:    prevention,  diagnosis,  treatment  or 
rehabilitation.    There  is  danger  in  using  tracers  to  look 
at  the  process  of  care  if  minimal  standards  for  medical 
management  cannot  be  agreed  upon. 

6.  The  effects  of  nonmedical  factors  on  the  tracer  should  be 
understood.    Social,  cultural,  economic,  behavioral  and 
environmental  factors  can  influence  the  prevalence  and 
distribution  of  many  diseases.    Thus,  the  epidemiology  of 
the  tracer  should  be  relatively  well  understood  and  the 
population  at  risk  easy  to  identify. 

Payne  in  his  study  of  hospital  c-:rc  ir,  Hawaii  evaluated  the  quality  of 
of  care  rendered  in  21  diagnostic  categories  which  represented  32 
percent  of  the  total  discharges  from  22  acute  general  hospitals  in 
Hawaii  (Tab  C).    Selection  of  these  conditions  was  based  primarily  on 
the  following  criteria: 

1.    The  condition  must  have  a  relatively  high  incidence. 
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2.  The  disease  or  conditions  should  not  be  a  self  limiting  one 
such  that  it  is.  not  responsive  to  therapeutic  manipulation, 

3.  It  should  not  be  unduly  dependent  upon  patient  cooperation, 

4.  Some  measures  of  preventive  care  or  at  least  early 
detection  should  be  demonstrated  to  affect  the  conditions. 

5.  The  diagnoses  selected  should  represent  care  delivered  by 
as  many  medical  and  surgical  specialties  as  possible, 

Williamson  has  developed  a  sophisticated  and  intricate  system  for 
measuring  the  quality  of  care  based  on  outcome  assessment,  Disease 
selection  is  based  primarily  on  the  estimation  of  expected  impairment 
and  the  preventable  impairment  for  each  diagnostic  category,  that  is, 
the  attainable  benefit  not  attained  (ABNA).    Expected  impairment 
includes  disability  (weighted  for  length  of  hospital  stay,  complications 
and  death),  and  social  disruption  (weighted  for  patient's  age  and 
frequency  for  which  a  disease  occurs).    Preventable  impairment  is 
estimated  by  physician  judgment,  using  available  literature,  to  deter- 
mine the  ability  of  good  medical  care  to  prevent  impairment.  In 
1968  Williamson  and  his  colleagues  (2)  implemented  this  system  in  a 
large  midwestern,  community  hospital.    By  applying  the  disability 
weights  (hospital  days,  complications  and  death)  a  list  of  ten 
priority  diseases  was  developed  for  this  hospital.    These  conditions 
and  physician  estimates  of  preventable  Impairment  are  found  in  Table  1 
below.    It  should  also  be  noted  that  Williamson  found  this  list  of 
priority  conditions  to  vary  from  hospital  to  hospital  depending  upon 
the  make  up  of  the  staff  and  the  focus  of  that  institution.  Estimation 
of  preventable  impairment  causes  great  difficulty  in  the  prioritizing 
process  as  there  exists  virtually  no  medical  literature 
for  comparison  (Williamson  searched  almost  11,000  articles  in  attempting 
to  set  standards  for  this  study's  estimation). 

Additional  criteria  are  used  in  eqidemiological  studies.    Among  the 
more  relevant  are: 

1.  The  disease  condition  must  be  validly  diagnosed. 

2.  In  mortality  statistics,  the  cause  of  death  must  be  reliably 
reported  on  the  death  certificate;  in  morbidity  statistics 
the  cause  of  hospitalization  must  be  accurately  separated 
from  other  concurrent  conditions. 

3.  The  onset  of  the  disease  and  its  health  impact  should  be 
separated  by  as  short  duration  as  possible,    (In  certain 
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disease  conditions,  e.q. ,  cancer,  "incubation"  periods 
from  15  to  20  years  have  been  demonstrated.    This  time 
lag  makes  the  assessment  of  the  impact  of  medical  care 
difficult  to  .iudqe.    Even  for  those  conditions  which 
have  often  been  used  in  the  assessment  of  morbidity  or 
mortality,  the  time  lag  is  considerable.    A  qood  example 
of  this  are  the  studies  on  the  impact  of  treatment  on 
hypertension  in  the  prevention  of  cerebrovascular 
accidents  (3)  which  required  follow-up  periods  of 
approximately  3  years), 

4,    The  denominator  populations  must  be  known  to  develop 
meaningful  rates  and  ratios. 

III.    Diagnosis  and  Procedure  Selection  Process 

The  process  of  selecting  the  diagnoses  and  procedures  for  which  utilization, 
quality,  and  cost  data  will  be  collected  and  analyzed  in  the  evaluation 
of  the  PSRO  program  utilizes  a  series  of  three  selection  criteria.  These 
were  developed  from  a  review  of  the  literature  and  expert  judgments.  The 

initial  three  selection  criteria  are: 

1,  Frequency  (high) 

2,  Health  System  Impact  (high) 

3C    Cost  of  Care  (high) 

If  a  diagnosis  (or  procedure)  has  a  high  frequency,  or  a  high  cost  or 
is  highly  responsive  to  the  health  care  system  (i.e.,  medical  care  " 
has  a  positive  impact  on  the  course  of  the  disease),  the  diagnosis  or 
procedure  is  then  characterized  to  determine  whether  1t  will  be  useful 
in  the  assessment  of  one  or  more  of  the  following  evaluation  areas: 

1 .  Quality  of  Care 

2.  Utilization  of  Resources 

3.  Cost  of  Care 

To  make  these  determinations,  characteristics  or  attributes  have  been 
developed  for  each  of  the  evaluation  areas. 

1.    The  total  set  of  Diagnoses  and  Procedures  useful  in  assessing 
quality  should  have  the  following  characteristics  (individual  diagnoses 

need  not  have  all  characteristics): 
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a,  High  health  system  impact,  i , e ,  the  natural  history  of 
disease  has  been  shown  to  vary  with  medical  treatment, 

b,  A  high  level  of  agreement  within  the  medical  profession 
regarding  the  appropriate  processes  of  care, 

c,  Clearly  identifiable  intermediate  or  long  term  outcomes, 

d,  Complications  identifiable  and  due  to  the  medical 
care  process, 

e,  Amenable  to  data  gathering  via  special  studies, 

f,  Increased  morbidity  or  mortality  if  services  are 
underutilized,  and 

g,  High  frequency  of  occurrence. 

2.  The  set  of  diagnoses  and  procedures  useful  in  assessing 
utilization  have  the  following  characteristics  (individual 
diagnoses  need  not  have  all  characteristics): 

a,  High  frequency  of  occurrence, 

b,  High  cost  of  care, 

c,  Diagnosed  with  a  high  degree  of  reliability, 

d,  Amenable  to  data  gathering  via  uniform  hospital 
discharge  data  set, 

e,  Stable  utilization  pattern  during  recent  time 
periods . 

3,  The  set  of  diagnoses  and  procedures  useful  in  assessing 
cost  have  the  following  characteristics: 

a.  Primarily  inpatient  and  limited  to  the  acute  short 
term  general  hospital  with  little  opportunity  for 
substitution  of  other  institutional  or  ambulatory 
care,  and 

b,  Representative  of  different  levels  of  cost  and 
frequency  of  occurrence. 

Finally  the  selected  diagnoses  should  be  representative  of  the  various 
medical  specialties, 

IV.    Summary  and  Recommendations 

Application  of  the  above  criteria  and  disease  characteristics  and  the 
technical  data  found  in  Tab  A,  yields  the  lists  of  diagnoses  (Tables 
2-4)  for  use  in  the  evaluation  of  the  PSRO  Program. 
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TABLE  2 

CONDITIONS  FOR  THE  EVALUATION  OF  QUALITY 
OF  CARE  IN  THE  PSRO  PROGRAM  EVALUATION 


ICDA-8 


1.  Hypertension 

2.  Pneumonia 

3.  Congestive  Heart  Failure  -  Pulmonary 
Edema 

4.  Urinary  tract  infection 

5.  Ulcer  stomach,  dudenal ,  peptic, 
gastrojejunal 

6.  Cholecystitis/Cholelithiasis 

7.  Inguinal  Hernia  without  obstruction 

8.  Hyperplasia  prostate 

9.  Hysterectomy 

10.  Intervertebral  Disc,  Displacement 

11.  Complications  &  adverse  reactions 


400  -  404 
480  -  486* 

427.0,  427.1 
590,  595 

531  -  534 
574,  575 
550 
600 

69.1  -  69.7 
725 

960  -  979,  995  -  999 


*Includes  unspecified  cases 
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TABLE  3 

CONDITIONS  FOR  THE  EVALUATION  OF  THE  UTILIZATION 
OF  SERVICES  IN  THE  PSRO  PROGRAM  EVALUATION 

ICDA-8 


1.  Cerebrovascular  Disease  430  -  438* 

2.  Chronic  Ischemic  Heart  Disease  412 

3.  Acute  Myocardial  Infarction  410 

4.  Diabetes  Mellitus  with  and 

without  acidosis  250 

5.  Malignant  Neoplasm  breast  174 

6.  End  Stage  Renal  Disease  580,  582, 


753.1,  759.8, 
590.0,  590.1, 

446.2,  752, 
753 


7.  Cholecystitis/Cholelithiasis  574,  575 

8.  Inguinal  Hernia  without  obstruction  550 

9.  Delivery  with  and  without  complications  650  -  662 

10.  Hypertrophy  Tonsils  and  Adenoids  500 

11.  Fracture  Femur  820 

12.  Displacement  Intervertebral  Disc  725 

13.  Cataract  374 

14.  Neuroses  300 


*Includes 


unspecified  cases 
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TABLE  4 

CONDITIONS  FOR  THE  EVALUATION  OF  THE 
COST  OF  CARE  IN  THE  PSRO  PROGRAM  EVALUATION 

ICDA-8 


1.  Cerebrovascular  Disease  430  -  438* 

2.  Pneumonia  480  -  486* 

3.  Acute  Myocardial  Infarction  410 

4.  Diabetes  mellitus  with  and  without 

acidosis  250 

5.  Appendicitis  540  -  542 

6.  Cholecystitis/Cholelithiasis  574,  575 

7.  Inguinal  Hernia  without  obstruction  550 

8.  Hyperplasia  Prostate  600 

9.  Hypertrophy  Tonsil  and  Adenoids  500 

10.  Fracture  Femur  820 

11.  Neuroses  300 


*Includes  unspecified  cases 


123 


TAB  A 

... 

Technical  Considerations  and  Data  Availability 


This  section  discusses  the  technical  considerations  related  to  the  use 
of  the  three  major  criteria-frequency,  system  impact,  and  cost.  The 
limited  data  indicate    a  level  of  caution  which  must  be  maintained 
throughout  the  development  of  the  program  evaluation, 

A,    Criteria:    Frequency  of  Diagnosis/Procedure 

Frequency  of  a  category  is  a  consistent  concern  in  the  evaluation 
analyses.    Three  data  sources  are  particularly  useful  for  further 
investigating  this  concern.    First,  the  National  Center  for  Health 
Statistics  published,  from  the  National  Health  Survey,  "Inpatient 
Utilization  of  Short-stay  hospitals  in  the  U.S.  in  1971"  (4). 
Tables  5  and  6  provide  a  summary  of  this  discharge  data  both  in  the 
aggregate  for  all  age  groups  and  for  the  age  group  over  65  years. 

A  second  source  is  Medicare  data  (5)  currently  available  for  discharges 
in  1969  and  1970.    These  data  include  the  following  parameters: 

1.  Number  of  Discharges 

2.  Average  stay  in  days 

3.  Standard  Deviation  of  Length  of  Stay  for  the  10th,  25th, 
50th,  75th  and  90th  percentiles. 

A  third  source  of  frequency  data  is  the  Medicaid  program.    No  national 
Title  19  utilization  data  are  available.    However,  data  reflecting  the 
rank  order  of  conditions  causing  hospitalization  in  the  states  of 
Illinois,  New  Mexico  and  Massachusetts  have  been  used  to  augment  the 
list  of  diagnostic  categories  frequently  encountered. 

The  utility  of  the  "frequency"  criteria  in  the  evaluation  of  the  PSRO 
program  depends  on  three  factors:    (1)  frequency  of  the  condition; 
(2)  number  and  validity  of  diagnostic  categories;    (3)  size  of  PSRO 
area. 

1 .    Frequency  of  the  Condition 

The  twelve  most  frequent  diagnostic  conditions  for  the  65  years  and 
older  population  are: 


124 


<T>  >X>  O  O  lO  i —  0> 
M        CSJ  PO  is.  CSJ  «J 


CO  CO  i— 


co       I©  CM  CO  N 

C  r*.      O  »j  co  in 


C  O        Csl  Oi  <7>  «t 
ID  O  (M  If)  O  M 


co  a* 

in  co  cm 


«3"  ^  CO 

a>  10  r~ 


O  r— 

o  in 


r-«.      i—  ir>  n      «f  <f  m 


LO  CJ\  CM 


co  m  cm  co     r-«*  co 

cn  O  T  CO       r-  Lf) 


co  r*.  m  cr.  i-*  cm  cm 


O  CT»  <uQ  CM  O  CM  O 


co  •—  o 


*t  O  in 
in  co  on 

CM  *T 


03 

Ci  O 


cm  in  ro  cm  ud  m  cm 


i      o     r-  in 


cn  o 


CO  co  cm  rs  n  m  in  in 
vo  into  IT)  CM  »-  r-  CO 
<—      iDuic\j  iOr-  in  cm 


r-  0% 

co  cr* 

CM  CM 


co  cm  co  co  co 
o%  CO  o     o  o 

CO  ^J-  CM       in  r- 


CM  m  m  CM  CO  *T  c*~> 
<T  CO  CO  (s  C  CM  C\J  ^ 
r—         in  r—  <—  CO  CM  lO  CM 


*r  cm 

CM  CO 


CO  Ol  CM 
r—  CO 
CM  ^  r- 


f—  o\ 

i£>  lO 

in  cm 


r—  lOCOOCJilOCOO 
r—  cn  co  r*.  co  ^  co 

CO         r—  VO  <J-  <}■  CO  r-  in 


r-  cm 
in  i© 


> —  CM  CO  m  CO 
CO  CO  O  l£N 

in  ^  cm     o  co 


to  CD 

c  o 


(O  - 

Ift 

cu  c 

E  **- 
3  C 


OS 

«3-  O 

CO 

ID  >£> 

o 

«3" 

Csj 

to  VO 

CO 

cn  r— 

o> 
o 

cn 

O  ro 

in 

VO  CO 

o 

m 

LO 

csj 

CM  o 

c 

o 

CXJ  CM 

CM 

*3- 

=3" 

to 

u) 

LT) 

to  10 

VO 

co  cn 

UI 

c 

o  o 

O  — 

1 

O 

1  1 

o  o 

O 

"9- 

i 

o 

t  i 
o  o 

«3"  ' — 

ro 

vo  CO 

m 

ro 

O  r» 

r—  csj 

■W  «T  «»■ 

tn 

io 

in 

id 

co  co 

CL 

CU 

*!  CU 

i    i  i 

1 

•  •  o 

i 

i  i 

E  x 

T3 

i    i  ■ 

t 

0)  0J 

1 

T. 

I 

i  i 

i  i 

CU 

4J 

CU 

UI 

*-> 

CM 

1  Co 

1  1 

t 

a; 

UI 

c  u> 

>.  c 

U>  » 

+-> 

4-1 

a; 

1    O  TJ 

1 

(/i  o 

1  1 

■o 

1 

><  E 

M- 

1 

i  i 

U» 

cn 

a, 

c 

U>  3 

*-» 

•a 

*j  or 

ID 

i  ■*->  u> 

>.*J 

1  1 

1 

C 

TO 

l 

LA 

I  c 

4-> 

ui 

u 

<-  u 

CU  CU 

at 

^, 

c 

■3 

o 

1  i-  -o 

O  CU 

>  -o 

O- 

3 

I 

*_» 

I  o 

c 

°i 

*o 

CU 

•l->  <*- 

•>-  o 

LA 

'i- 3 

CU 

'  i 

<U  «*_  +J 

%/i 

*J  c 

1  U) 

+-»  3 

c: 

CU 

i 

t 

UI 

UI 

X 

LA 

w>   C  L. 

rO 

I- 

E 

UI 

UI  T3 

3 

•f—J 

"o 

«3 

c 

CU  c 

CU 

E 

ro  i-  ia 

0j 

.  C 

CO 

o 

i 

l 

-C 

3 

o 

*j  cu 

ui 

■5  "o 

la 

(1)  CO 

i/i 

Q. 

o 

o 

cn  - 

u 

UI  L- 

•«-  a. 

cu 

ui 

fO  iA 

<—  -C 

UI  w 

i  «- 

J= 

o 

•i-  ■»-> 

IA  O 

t_ 

t-  *? 

O 

OJ 

■—  E 

1/) 

•r-  fO 

■o 

CU  o 

-a  o 

u^ 

re 

4->  UI 

o 

o  £• 

tA 

a.  u> 

3 

T3  •»-  U 

-  -- 

<o 

l_ 

w. 

C 

■o 

■—  T3 

it-  ro 

ro 

O  «D 

■f-> 

T5 

o 

CO  E 

<u 

cn 

c; 

cn  cu 

c 

r—  C 

aj 

ID  — 

*->   1-  E 

CU  1- 

i  «j 

1 

-c:  o 

u 

w 

UI 

C  E 

o 

ro  ro 

fO 

ro  "» 

■a 

{A 

C  CL 

w  re  oj 

.c 

>. 

4->  •*-> 

■o 

co 

<T3 

47 

u 

c 

o 

<D   U  -C 

<->  Q. 

rsj  » 

-C 

l/l 

U> 

3 

c 

.c 

■—  t- 

•  C 

O 

■a 

*j  ai 

0) 

(you 

U 

UI 

c  >o 

Q.-0 

^1 

.C 

O  O 

UI  o 

■o 

u 

c  c 

E 

>A 

<*-  CU 

CU 

o 

O  vi- 

ra 

CU  •- 

UI 

ro 

E  — 

O  u 

3  C 

o 

o 

!_ 

(O 

u  ui 

o  •• 

t.  *J 

UI 

fa 

E 

cr  c 

la 

U 

> 

t—  O 

4-» 

(/> 

CU 

Cu 

c 

(. 

3  « 

CU 

cu 

UI 

cn  cn 

Cu 

•r-    OI  !. 

o 

CU   CU  X-  E 

1_ 

01 

CU  i- 

a.  j-> 

<-> 

CD 

T3  CO 

Z  CO 

■J  U 

■D 

-C 

*J 

E  w  01 

t- 

UI  4-> 

C  3 

<U 

T3 

u>  CU 

CU 

3 

C  -D 

(J  CU 

3 

Is 

i_ 

1—  c 

aj 

-O 

«  3 

..-  CO 

Q. 

T3 

Q. 

UI 

3 

C"  O 

«  L- 

CU  f- 

ro  U 

i- 

a 

A3  cu 

-O 

LT    ^  .:.  ' 

tl) 

CU  u 

c 

>s 

CO  — 

C 

O 

•!->  3 

i-  ro 

U 

•■o 

OEO 

<T3 

O  «t  O 

1- 

u>  <£ 

D. 

X 

UI  13 

(_> 

u-  — 1 

c 

=  5 

CU 

ai 

o  z 

<_>  o 

5 

a 

o— • 

CSJ  * 

125 


>>  i— 


C  r— 


to  CM 

CO  CTn 


r-*.  ct^  cm  i —  rs  tj-  c 
m  cm  r*-  cm  m  cm  ^—  n  cm  cm 


CNJ  CVJ  CO  UJ 

«t  n  i-  rs. 


Ol  CM  r—  CO 

n      ro  co 


co  cm  in  n  (\j  . —  m«3-u3csj 


cr>  <— 


r^.     I^r  <o  • —  in  co  tJi  o  «j 


CMC0uDCOntTC0CO--«3- 

^xuDr^cO'-NkoaM/ip — 

CO  r—  m        >—  I— 


CO  «3"  O  N 


C  N  CO  ^)  N  O  ' —  UD  n  ID 


«=f  CM  lO 


in  co  m  r*. 


O^*C0^3*CMCNJC0'^l-0r»s> 

<T  io^r-^ir)i-  r>»  cn  rs 

UDr-  r-CO»—  0<—  CO  f— 


O  C\J  co  cn 

LO  co  CO  C71 
r-r-  r-  CM 


cno>t  CO«tOC0Olfi 

Ocnr^f— r— r-    •  co  «3-  co 

I  I  I   CM    I  I 

O  O        p—  O 


r—  •   CO 


CO 


LT)  O  CO  CTi 

p**  o  i—  cm 
lo  «x>  co 
*       i  i 

«3"        O  O 

—  o 

in      r-*  co 


4j  4_  +J  3  i/l 


i-  TO  3 

4-»  ■ —  +-> 

-O  O  — 

C  OJ  — 


C  t/1  -4->  U 

to  aj  o  ••- 

CT1  OJ 
— 

i —  TO 

TO  -r- 

3:  a  i 


3  -C 

■a  o 


-*->  l/l 
V.  1- 

OJ  ■—  O 
JZ  3  L. 

>  TO  u 
1-  >  Ul 
*->  O  O 
i/i  I.  — 
_  p  .c  - 

DUO) 
C  i-  4J 
Oil. 

o  o  «t 


(1)  E  Q..— 
i —  XZ   O   l/l  TO 


3   3  • 


l/l  l/l 

S  a; 


TO   l/l   O  ■—  I— 


TO 


C   «   L   1.  L 

O  (A  9 

E  fl  U  U  M 


'r   Ol  ifl   (fl  ifl 

f—   C   TO  •<-  OJ 

QJ    TO  ' —  *■>  l_ 

—  r—    Q.—  3 

Z    C     -    -  -' 

££   U£  U 

U  U  O.  -h->  TO 
>.!-!- 


l/l  f-  Q£ 


X>  T3  i— 
l/l    l/l  TO 


—  —  < 


126 


3 , 

Lnron ] c  nf CBri osc i ero cue  near  t  ui scqse 

h 

LGrGDra I    VaSCU 1 ar  UlScaSc 

IT* 

r  ncuinun  8  dl 

A 

Q . 

rraC  tue  e  OT  trie  s ower  1  iniD 

e , 

nGrnla  WltnOUL  ous true l i on 

■p 

T  , 

la  Las  aL I 

g. 

Diabetes  mellitus 

h. 

Acute  myocardial  infarction 

i . 

Hypertension 

j. 

Hyperplasia  Prostate 

k. 

Cholelithiasis 

1. 

Ulcer;  stomach,  duodenum  and  jejunum 

From  other  available  (e.g.,  NCHS  all  ages)  data,  the  following  three 
conditions  could  be  added: 

a.  Uncomplicated  delivery 

b.  Hypertrophy  of  tonsils  and  adenoids 

c.  Menstrual  disorders. 

Frequent  operations  and  procedures  could  also  be  considered.  The 
National  Center  for  Health  Statistics  has  published,  "Surgical 
Operations  in  Short-Stay  Hospitals,  United  States  -  1971"  (6)  which 
provides  frequency  data  on  the  occurrence  of  major  operations  and 
major  procedures.    The  ten  most  frequent  are: 

a.  Tonsillectomy  with  or  without  adenoidectomy 

b.  Dilation  and  curettage  of  uterus,  diagnostic 

c.  Biopsy 

d.  Hysterectomy 

e.  Repair  of  inquinal  hernia 

f.  Excision  of  lesion  of  skin  and  subcutaneous  tissue 

g.  Cholecystectomy 

h.  Oophorectomy;  salpingo-oophorectomy 

1 .  Appendectomy 

j.  Closed  reduction  of  fracture  without  fixation 

Several  points  are  noted  in  considering  these  data:    1)  minor  procedures 
e.g.,  cystoscopy  are  subsumed  under  more  major  procedures,  2)  categories 
are  not  precisely  divided  but  rather  represent  the  lumping  of  many 
specific  procedures  and  3)  while  the  incidence  of  certain  procedures 
is  extraordinarily  high  on  a  national  basis  the  incidence  in  some  PSRO 
areas  may  not  be  very  high. 

The  difficulties  encountered  in  a  precise  division  of  the  categories 
is  illustrated  by  category  2988,  cardiac  revascularization,  which 
includes  15  different  procedures,    While  365,000  events  are  indicated 
for  this  category,  it  is  impossible,  at  this  time,  to  obtain  specific 
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frequency  data  on  the  coronary  by  pass  procedure;  others  (7)  have 
estimated  that  20,000  coronary  by  pass  operations  were  performed  in 
1971  rather  than  the  larger  number  suggested  by  these  data, 

2 ,    Number  an_d_  Va  1  idity  of  Diagnostic  and  Procedural  Categories 

While  the  previously  mentioned  frequent  diagnoses  and  procedures 
appear  to  have  a  very  high  incidence,  care  must  be  taken  to  consider 
the  number  of  different  conditions  within  each  ICDA  grouping  and  the 
validity  of  these  diagnoses,    The  ICDA  three  digit  groupings  for 
pneumonia,  all  forms  (480-486)  and  cerebrovascular  disease  (430-438) 
serve  as  excellent  examples  of  this  concern. 

Pneumonia  is  disaggregated  into  more  than  19  other  etiological 
conditions.    The  National  Center  for  Health  Statistics,  Hospital 
Care  Statistics  Branch  has  further  detailed  the  frequency  of  this 
condition.    (See  Table  7  below) 


TABLE  7 


Code  and  Diagnosis 


Number  1971        Percent  of  all 

Discharges  (Thousands)    Discharges  (all  ages) 


480  -  Viral  pneumonia  32 

481  -  Pneumococcal  pneumonia  136 

482  -  Other  Bacterial  pneumonia  12 

483  -  Pneumonia  due  to  other 

organisms  8 

484  -  Acute  Interstial  pneumonia  3 

485  -  Bronchopneumonia,  unspecified  166 

486  -  Pneumonia,  unspecified  284 


<0.10 
0.46 

<  0.10 

<  0.10 
<0.10 

0.56 
0.96 
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Thus  a  condition  which  at  first  glance  appears  to  be  extremely  useful  as 
a  measure  of  health  care  effectiveness,  on  further  investigation  shows 
3  of  4  cases  insufficiently  diagnosed  (as  indicated  by  the  discharge 
front  sheet  and  at  this  time  there  is  no  indication  the  PSRO  discharge 
data  will  be  anymore  detailed)  to  be  of  use  in  rigorous  evaluation 
studies.    Further  study  will  be  required  to  determine  what  proportion 
of  the  unspecified  pneumonias  actually  had  a  specified  etiology 
recorded  in  the  chart  (were  false  negatives)  and  what  proportion  of 
the  specified  pneumonias  did  not  have  a  specified  etiology  recorded 
(were  false  positives). 

The  extent  of  these  false  positive  and  false  negative  diagnoses  rather 
than  preventing  the  use  of  particular  diagnoses  more  appropriately 
requires  careful  use  of  PSRO  data  in  answering  specific  questions. 
For  instance,  utilization  and  cost  questions  regarding  "pneumonia" 
will  provide  useful  and  valid  information  when  the  general  condition 
of  "pneumonia"  is  correctly  diagnosed.    If  the  question,  however, 
relates  to  the  quality  of  care  provided  patients  with  "pneumococcal 
pneumonia,"  measurement  and  control  of  misclassif ication  error  must 
be  achieved.    Fleiss  (8)  has  reviewed  misclassification  errors  and 
suggests  methods  for:    control  of  such  errors,  estimating  the  magnitude 
of  such  errors,  and  determining  their  statistical  significance,  In 
general  these  methods  require  independent  reclassification  of  a  sample 
by  one  or  two  individuals,  and  while  such  techniques  are  time  consuming 
and  expensive,  they  are  an  essential  part  of  investigating  specific 
evaluation  questions.    The  reclassification  process  can  also  be  an 
educational  one  as  a  review  of  the  75%  of  the  pneumonias  which  are 
unspecified  might  lead  to  the  use  of  additional  diagnostic  tests  or 
deletion  of  inappropriate  therapies. 

Cerebrovascular  disease  suggests  additional  complexities.    First,  there 
are  approximately  88  subclassifications  of  this  condition.    Table  8  below 
provides  an  analysis  of  disaggregate,  NCHS,  Hospital  Discharge  survey 
data: 


TABLE  8 


Code  and  Diagnosis 


Number  1971        Percent  of  all 

Discharges  (Thousands)    Discharges  (all  ages) 


430  -  Subarachnoid  Hemorrhage 


18 


0.10 


431  -  Cerebral  Hemorrhage 


38 


0.12 


432  -  Occlusion  precerebral  arteries 


28 


0.10 


433  -  Cerebral  Thrombosis 


81 


0.27 
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TABLE  8  (cont.) 


Code  and  Diagnosis 


Number  1971        Percent  of  all 

Discharges  (Thousands)    Discharges  (all  ages) 


434  -  Cerebral  embolism 


5 


0.10 


435  -  Transient  cerebral  ischemia 


26 


0.10 


436  -  Acute  but  ill-defined 

cerebrovascular  disease 


200 


0.68 


437  -  Generalized  ischemic 

cerebrovascular  disease 


108 


0.36 


438  -  Other  and  ill  defined 

cerebrovascular  disease 


25 


0.10 


Sixty-three  percent  (333,000  of  529,000)  of  the  cases  are  characterized 
as  "ill-defined"  or  "generalized". 

Second,  in  utilizing  cerebrovascular  disease  it  will  be  necessary  to 
control  comparative  PSR0  evaluations  to  account  for  variations  in 
incidence  and  case-fatality  rates.    The  Nationwide  Cerebrovascular 
Disease  Morbidity  Study  (9)  documented: 

a.  A  2-3  fold  difference  in  incidence  between  low  incidence 
areas  (Florida,  Colorado  and  Kansas)  and  high  incidence 
areas  (Georgia,  North  Carolina  and  South  Carolina); 

b.  A  28%  case  fatality  percentage  in  Denver  and  a  49%  case 
fatality  percentage  in  South  Carolina; 

c.  A  difference  in  length  of  stay  reflecting  the  higher  case 
fatality  rates  in  the  southeast. 

Thus  before  selecting  any  condition  for  the  evaluation  of  the  PSR0 
program  an  ajustment  must  be  made  for  the  validity  of  available 
diagnostic  data  and  natural  variations  of  incidence  and  mortality 
which  might  occur  nationally.    This  again  emphasizes  the  need  for 
accurate  baseline  data. 

3.    Size  of  PSRO  Area 

The  size  of  the  PSRO  area,  as  measured  by  number  of  expected  Title  18 
and  19  admissions,  could  significantly  limit  the  number  of  diagnoses 
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and  procedures  that  could  be  considered  for  use  in  the  evaluation.  For 
example,  consider  a  PSRO  area  with  20,000  or  less  admissions  per  year. 
A  high  frequency  diagnostic  condition  such  as  cholelithiasis  (11th 
most  frequent  condition  in  the  65  and  old  age  group)  would  account  for 
less  than  400  admissions  annually  in  that  PSRO  area.    There  are  more 
than  20  PSRO  areas  with  20,000  or  less  Medicare  and  Medicaid 
admissions  per  year. 

While  the  actual  number  of  cases  necessary  to  discriminate  between 
PSROs  will  depend  on  the  questions  asked,  and  the  methods  and 
statistical  tests,  some  estimation  can  be  obtained  from  previous 
studies. 

Prospective  studies  are  the  most  demanding  in  regard  to  sample  size 
required  and  thus  establish  an  upper  limit  for  numbers  required.  If 
the  incidence  rate  (not  hospitalization  rate)  of  a  condition  for  the 
period  of  study  is  1/100  and  the  risk  between  the  PSRO  area  and  non- 
PSR0  area  is  2,  the  number  of  cases  required  would  be  1,000  to  establish 
an  80%  probability  of  detecting  the  difference  with  the  significance 
level  at  0.05.    This  describes  the  extreme  lower  limit  of  a  sample 
size  for  a  prospective  study.    The  size  required  will  often  be  much 
higher  because  the  incidence  is  often  not  as  high  as  1/100  and 
anticipating  a  risk  of  2  in  non-PSRO  vs  PSRO  areas  is  rather  optimistic. 
Retrospective  studies  could  require  as  few  as  20-40  carefully 
documented  cases,  and  the  use  of  non-parametric  statistics  can  define 
extreme  statistics  with  even  fewer  cases.    If  an  evaluation  is  to 
include  all  PSRO  areas  or  use  a  stratified  sample,  consideration 
should  be  given  to  combining  these  small  areas  with  adjacent  areas, 
and  thus  provide  for  a  larger  number  of  admissions.    This  topic  will 
merit  major  consideration  as  methodologies  are  decided  upon. 

B.    Criteria:    Health  System  Impact 

The  diagnostic  categories  contained  in  Table  9  represent  those  in  which 
the  health  system  could  have  a  high  positive  impact  on  patient  outcome 
as  identified  by  Williamson,  Kessner,  the  Kellogg  Private  Initiative 
Study  of  PSRO,  and  0PSR  staff. 

A  review  of  this  table  indicates  that  the  following  conditions  are 
recommended  by  more  than  one  source: 

1.  Pneumonia, 

2.  Acute  myocardial  Infarction, 

3.  Delivery  (complicated  and  uncomplicated), 


4. 


Cholel ithiasis, 
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Table  9  -  Conditions  on  which  the  health  care  system  has  substantial 
positive  impact. 

Program                   Williamson  Kellogg 
Condition  Estimation  (Rockford)  Kessner       PI  in  PSRO 


1.  Pneumonia  x 

2.  Acute  Myocardial 

Infa ret ion 

3.  Cataract  x 

4.  Diabetes  Mellitus 

5.  Hernia  without 

Obstruction  x 

6.  Cerebral  Vascular 

Disease 

7.  Hyperplasia  prostate 

x 

8.  Delivery,  Com- 

plicated and 
uncomplicated  x 

9.  Cholelithiasis  x 

10.  Fracture  Lower 

Limb  x 

11.  Lacerations, 

trauma  x 

12.  Acute  Appendi- 

citis x 

13.  Urinary  Tract 

Infections 

14.  Complications 

and  Adverse 
Reactions  x 

15.  Hypertension  x 

16.  Displaced  inter- 

vertebral 
Lumbar  Disc 

17.  Hemorrhoids 

18.  Emphysema 

19.  Otitis  Media 

with  Hearing 
Loss 

20.  Visual  Disorders 

21.  Iron  Deficiency 

Anemia 

22.  Cervical  Cancer 

23.  Diarrhea  in 

children 

24.  Upper  G.I.  ulcer 

Disease 


Note:    Kessner  was  also  considering  outpatient  conditions. 


132 


5.  Fracture  of  the  lower  limb, 

6.  Urinary  tract  infections,  and 

7.  Hypertension 

Other  conditions  also  merit  special  considerations.    For  example,  end 
stage  renal  disease  has  become  a  prominent  and  increasing  cause  of 
admission  to  short  term  general  hospitals  subsequent  to  the  passage 
of  the  1972  Social  Security  amendments.    The  impact  of  this  legislation 
on  hospital  utilization  data  is  not  yet  available,  but  anecdotal 
information  suggests  that  patients  with  chronic  renal  disease,  and  its 
complications  could  constitute  the  third  most  common  cause  of  short 
term  hospital  admission  (in  hospitals  with  active  chronic  renal  disease 
programs).    This  increase  in  utilization  along  with  the  concern  for 
the  improved  quality  of  life  make  this  an  important  PSRO  quality  of 
care  concern. 

Another  category  of  special  consideration  is  that  of  adverse  drug 
effects.    Jick,  in  "Drugs-Remarkably  Nontoxic"  (10)  reports  that 
the  Boston  Collaboratine  Drug  Surveillance  Program  has  found  30 
percent  of  medical  inpatients  have  an  adverse  drug  reaction  with  the 
most  common  reactions  being  nausea,  drowsiness,  diarrhea,  vomiting 
and  rash.    Three  percent  of  patients  (n=19,000)  had  life  threatening 
reactions  and  46  deaths  occurred  or  2.4  per  1,000.    Since  each  patient 
received  an  average  of  nine  different  drugs  during  the  hospitalization, 
individual  drugs  were  considered  to  be  remarkably  non-toxic  and,  in 
fact,  16  deaths  were  due  to  excessive  administration  of  potassium 
chloride  and  8  from  excessive  fluid  administration. 

C.    Criteria:    Cost  of  Care  for  Diagnostic  Condition  or  Procedure 

While  cost  data  are  copiously  available  for  medical  care  payment 
programs  such  as  Title  18,  Title  19,  Blue  Cross  etc.,  (See  SSA-0RS 
publication  73-11903,  "Compendium  of  National  Health  Expenditures 
Data")  they  are  conspicuously  absent  for  specific  diagnostic 
categories.    Medicare  data  for  1969  (11)  provide  "mean  charge  per 
Discharge",  but  these  data  reflect  almost  entirely  mean  charge  per  day 
times  the  length  of  stay  taking  into  account  only  the  presence  or 
absence  of  surgery.    They  are  useful  however  in  providing  the  relative 
relationships  of  costs,  one  condition  to  the  other.    Table  10  provides 
a  synopsis  of  these  data. 

These  limited  cost  data  suggest  that  for  the  1969  Medicare  population 
the  most  costly  conditions  are: 

1.    Fracture  of  the  neck  of  femur,  closed 


2. 


Acute  coronary  occlusion 
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Table  10-  Standardized  Mean  Charge,  1.00=1969  Average  Cost  of 
Hospitalization  for  a  Medicare  Beneficiary 


Standardized 

Primary  Discharge  Diagnosis  ICDA  7th        Mean  Charge  per 

Discharge 


Malignant  neoplasm  of  breast 

170 

i 

OO 
.  CO 

Malignant  peoplasm  of  prostate 

177 

i 

1  c 

Malignant  neoplasm,  generalized 

199 

i 

01 

Diabetes  mellitus 

260 

n 
u 

.  yo 

Cerebral  hemorrhage,  nontraumatic 

331 

i 

i  n 
.  1  u 

Cerebral  embolism  and  thrombosis 

332 

i 

90 

Other  and  ill-defined  vascular  lesions  affecting 

central  nervous  system 

OO/I 

n 
u 

PQ 
.  OO 

cataract 

OQC 
JOD 

n 
U 

.  Oc. 

Arteriosclerotic  heart  disease  so  described 

ft  on  n 
QcX)  .(J 

n 
U 

.  yo 

Acute  coronary  occlusion 

/ion  l 
<+<:U .  1 

1 
1 

AA 

Other  heart  disease  specified  as  involving 

coronary  arteries 

n  on  "3 

n 
u 

QC 

.  yo 

Other  myocardial  degeneration,  excluding 

fatty  degeneration 

A  OO    1       /IOO  Q 

n 
u 

QC 

.  yo 

utner  ana  unspeciriea  diseases  ot  neart 

A  0/1 

i 

m 

.  U  1 

Other  hypertensive  heart  disease 

u 

on 

.  by 

Other  hypertensive  disease 

A/17 

n 
u 

.  D  / 

Arteriosclerosis  not  further  specified 

a  cn  n 

n 
u 

QQ 

.  oy 

Bronchitis,  acute  and  unqualified,  and 

dLULc    Upper    i  csp  1  f  a  LUi  Jf  MllcLUIUil 

475  5nn  sni 

n 
U 

CO 

Pneumonia 

490-493 

n 
u 

qc 
.  yo 

Chronic  bronchitis  and  emphysema  without 

mention  of  bronchitis 

502,527.1 

n 
u 

.  y  o 

Ulcer  of  duodenum  without  perforation  and 

without  hemorrhage 

541.0 

n 
u 

RA 
.  Of 

Hernia  of  abdominal  cavity  without 

mention  of  obstruction 

560 

o 

80 

Intestinal  obstruction  without  mention 

of  hernia 

570 

1 

.17 

Gastroenteritis  and  colitis 

571 

0 

.53 

Di  verticul i tis 

572.1 

0 

.81 

Choi  el i thiasis 

584 

1 

.26 

Cholecystitis  and  cholangitis  without 

mention  of  calculi 

585 

1 

.01 

Pyelitis,  pyelocystitis,  and 

pyelonephritis 

600.0 

0 

.86 

Hyperplasia  of  prostate 

610 

1 

.20 

Fracture  of  neck  of  femur,  closed 

820.0 

1 

.88 

All  other  diagnoses 

0 

.96 
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3.  Malignant  Neoplasm,  with  matasteses 

4.  Cholelithiasis 

5.  Malignant  Neoplasm,  breast 

6.  Cerebral  Embolism  and  Thrombosis 

7.  Intestinal  obstruction  without  hernia 

8.  Hyperplasia  of  prostate 

9.  Malignant  neoplasm  of  prostate 

10.  Cholecystitis  and  cholangitis  without  calculi 

"Relative  Cost"  data  were  also  determined  by  standardizing  length  of 
stay  data  for  the  given  disease  or  group  of  diseases  against  the  national, 
all  age,  average  length  of  stay  in  1971  (7.80  days).    Those  conditions 
with  a  stay  of  150%  or  greater  than  the  average  were  defined  to  be 
high  (H)  cost  disease  conditions.    Those  less  than  50%  of  the  average 
were  defined  as  low  (L)  cost  conditions.    Those  from  50%  -  150% 
were  defined  as  medium  (M)  cost  diseases.    Using  this  measure  the 
following  conditions  were  considered  high  cost  conditions: 


1 
2 
3 

5 
6 
7 
8 
9 

10 


Carcinoma  of  the  Breast 

Neoplasm  of  large  intestine  and  rectum 

Chronic  Arteriosclerotic  Heart  Disease 

Acute  Myocardial  Infarction 

Diabetes  Mellitus 

Cerebral  Vascular  Disease 

Hyperplasia  of  Prostate 

Cholel ithiasis 

Fracture  of  the  lower  limb 

Psychoses 


Table  11  is  a  compilation  of  the  above  frequency,  cost  and  quality  data. 


Table  1' 
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COMPILATION  TABLE 


PERCENTAGE  OF  DISCHARGES 


icoa-8 

CODE 


NCHS 
ALL  AGES3 


NCHS  65 
YRS  &  OLDER3 


RA5K 
ORDER 
XIX4 


ESTIMATED 
HEALTH 
SYSTEM 
IMPACT 
RELATIVE  ON 


COSTS- 


OUTCOME 


Diarrheal  Diseases 

009 

1.05 

0.S1 

L 

M 

Beoplasm,   Lge  Intestine 

153,154, 197,5 

0.43 

1.18 

B 

M 

4  Rectum 

Neoplasm,  Breast 

174 

0.47 

0.82 

H 

M 

Diabetes  Mellitus 

250 

1.45 

2.8 

2.74 

(8) 

H 

L 

Psychoses 

290-299 

0.81 

0.49 

H 

M 

Cataract 

374 

0.81 

3.0 

2.81 

M 

B 

Bypertens  ion 

400-404 

0.76 

2.35 

a 

Acute  Myocardial 

410 

1.14 

3.2 

2.70 

H 

M 

Infarction 

Chronic  Arteriosclerotic 

412 

3.29 

4.7 

9.73 

H 

M 

Heart  Disease 

Other  Coronary  Heart 

411,413,414 

0.58 

2.2 

1.15 

(12) 

H 

N 

Disease 

Heart  Disease  -  Unspec- 

420-429 

0.82 

2.7 

1.73 

H 

M 

ified 

Cerebral  Vascular 

430-438 

1.79 

5.6 

6.30 

H 

M 

Disease 

Upper  Respiratory 

460-465 

1.19 

0.52 

(6) 

M 

L 

Infections 

Pneumonia 

480-486 

2.17 

3.3 

2.96 

(2) 

M 

H 

Hypertrophy  Tonsils  & 

500 

3.32 

* 

(4) 

L 

L 

Adenoids 

Dicer;  Stomach. 

531-534 

1.42 

1.75 

K 

M 

Duodenum,  Jejunum 

Acute  Appendicitis 

540-543 

0.96 

0.19 

(11) 

M 

B 

Hernia  without 

550-553 

1.59 

2.6 

2.84 

M 

B 

Obstruction 

Gastroenteritis 

561 

0.55 

0.83 

(3) 

N 

N 

Cholelithias  is 

574 

1.36 

1.7 

1.76 

B 

H 

Cholecystitis 

575 

0.44 

0.8 

0.73 

(5) 

N 

M 

Urinary  Tract  Infections 

590,545 

1.03 

1.38 

n 

M 

Hyperplasia  Prostate 

600 

0.64 

2.4 

2.19 

B 

H 

Menstrual  Disorders 

626 

1.47 

0.16 

(10) 

M 

K 

Pregnancy  witn  complic- 

bJO-b34 

1.06 

L 

H 

ations 

False  Labor 

634.7 

L 

L 

Abortion 

64  0-6  4  5 

1.60 

(7) 

L 

H 

Uncomplicated  Delivery 

650 

9.40 

(1) 

M 

H 

Delivery  with  Complic- 

651-661.9 

1.92 

H 

B 

ation 

Fracture,  All 

800-829 

3.62 

4.89 

H 

H 

Fracture  Lower  Limb 

820-829 

1.03 

2.87 

3 

H 

Lacerations,  Trauma 

870-907 

1.26 

0.47 

M 

B 

Complications  and 

960-999 

0.91 

1.11 

(9) 

H 

B 

Adverse  Reactions 


BOXES:     1.   "NCHS  all  ages"  are  National  Center  for  Health  Statistics  data  adjusted  to  percent 
of  total  discharges  for  all  ages.     These  data  are  from  a  sample  of  379  hospitals, 
highly  stratified  on  the  basis  of  bed  sizes  and  representative  of  the  6,965  short 
stay  hospitals  in  the  U.S.   in  1970. 

.  2.  "XVIII"  is  1969  data  from  the  Office  of  Research  and  Statistics,   Social  Security 
Administration,   expressing  in  percent  of  the  total,   the  discharges  for  the  1969 
Medicare  population. 

3.  "NCHS  65  years  and  older"  are  1970  data  from  the  National  Center  for  Health  Statistics 
derived  from  the  all  ages  data. 

4.  "XIX"  data  are  a  compilation  of  anecdotal  information  from  the  states  of  New  Mexico, 
Illinois  and  Massachusetts  indicating  in  rank  order  the  most  frequent  causes  for 
hospitalization  in  those  programs. 

5.  H,  M,   and  L  indicate  high,  medium,   and  low  costs. 

6.  H,  H  and  L  indicate  high,  medium  and  low  amounts  of  positive  impact  by  the  health  care 
system  on  the  outcome  of  the  disease  or  condition. 


*     Data  does  not  meet  standards  of  reliability  or  precision...     Not  applicable. 
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TAB  B 

Tracer  Diseases  as  Defined  by  Kassner  et.  al . 

1.  Otitis  media  with  hearing  loss 

2.  Visual  disorders 

3.  Iron  deficiency  anemia 

4.  Hypertension 

5.  Urinary  tract  infections 

6.  Cervical  cancer 
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TAB  C 

Diagnostic  Categories  Utilized  by  Payne  in  Assessing  the  Quality 

of  Hospital  Care  in 
Hawai  i 


Acute  Cholecystitis 
Chronic  Cholecystitis 
Cancer  of  the  Breast 
Congestive  Heart  Disease 
Pregnancy  Full  Term 
Ceasarean  Section 
Abortion 
Pneumoni  a 
Bronchitis 

Cerebral  Vascular  Accident 
Stroke 

Diabetes  Mellitus  New 
Diabetes  Mellitus  Old 
Fi  bromyoma 
Gastroenteritis 
Fracture  Radius  and  Ulna 

Benign  Prostate  Hypertrophy  and  Cancer  Prostate 
Cancer  of  the  Cervix 
Tonsillectomy  and  Adenoi dectomy 
Urinary  Tract  Infection,  Acute 
Urinary  Tract  Infection,  Chronic 
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APPENDIX  B 
COLLECTION  OF  BASELINE  DATA 
FOR  PSRO  EVALUATION 


I .  Introduction 

An  extensive  and  continuing  evaluation  of  the  PSRO  program  is 
necessary  to  assure  its  successful  growth  and  development.  The 
accomplishment  of  this  task  is  dependent  upon  the  ability  to 
collect  baseline  data.    For  purposes  of  this  paper,  baseline 
data  are  defined  as  information  pertaining  to  the  period  of  time 
prior  to  implementation  of  a  PSRO. 

These  baseline  data  will  be  utilized  by  individual  PSROs  for 
identifying  specific  problem  areas  requiring  extensive  review 
efforts  and,  in  later  periods,  evaluating  their  impact. 

At  the  National  level  baseline  data  will  be  used  to  evaluate  the 
impact  of  the  PSRO  program  on  physician  and  institutional  behavior 
as  it  relates  to  quality,  utilization  and  cost.    The  following 
evaluation  issues  specified  by  the  plan  require  collection  of 
baseline  data: 

1.  Has  Admission  Certification  and  Continued  Stay  Review 
(concurrent  review)  been  effective  methods  of  reducing 
medically  unnecessary  hospital  utilization? 

2.  Is  Concurrent  Quality  Assessment  (CQA),  as  carried  out  in  the 
PSRO  program,  an  effective  method  of  assuring  the  appropriate 
utilization  of  services  (diagnostic  and  therapeutic  proced- 
ures)? 

3.  Is  retrospective  health  care  review,  as  carried  out  in 
the  PSRO  program,  an  effective  method  of  assessing  the 
quality  and  appropriateness  of  the  utilization  of  services? 

4.  Are  the  health  care  review  activities  (AC,  CSR,  MCE,  and 
profiling),  as  carried  out  in  the  PSRO  program,  assuring 
the  quality  and  appropriateness  of  the  utilization  of  health 
care  services? 

5.  What  has  been  the  impact  of  PSRO  activities  on  physicians' 
behavior? 

6.  What  has  been  the  impact  of  PSRO  activities  on  institutional 
behavior? 
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7.  What  has  been  the  effect  of  PSRO  activities  on  health 
expenditures  in  the  U.S.? 

8.  What  has  been  the  impact  of  PSRO  activities  on  the  health 
status  of  the  Medicube  and  non-Medicube  populations? 


II .    Types  of  Data 

Baseline  data  at  the  National  level  can  be  classified  into  the 
following  types: 

1.  Utilization  review  data  (length-of-stay  and  admissions  by 
diagnosis) 

2.  Quality  of  care  data  (diagnostic  and  therapeutic  procedures 
as  related  to  patient  management  and,  to  the  extent  feasible, 
measures  of  medical  outcome) 

3.  Cost  of  review  and  cost  of  care  data 

4.  Population  data. 

Utilization  Review  Data 

Data  on  admissions  and  length  of  stay  by  diagnosis  could  be  collected 
for  a  national  sample  of  hospitals.    Because  number  of  admissions  should 
be  related  to  population  bases,  it  would  be  desirable  to  collect  data 
for  all  hospitals  in  individual  PSRO  areas  for  a  sample  of  PSROs.  In 
cases  where  this  is  not  feasible,  an  assumption  could  be  made  that 
changes  in  admission  rates  by  diagnosis  do  not  reflect  shifts  in 
admissions  between  hospitals  in  and  out  of  the  sample. 

Quality  of  Care  Data 

For  the  collection  of  quality  of  care  data  (diagnostic  and  treatment 
procedures  and,  to  the  extent  feasible,  measures  of  medical  outcome), 
a  basic  question  exists  as  to  whether  the  collection  of  baseline 
data,  for  specific  diagnoses,  from  a  national  sample  of  hospitals 
is  appropriate.    Changes  in  diagnostic  and  treatment  modes  are  likely 
to  result  from  MCE  studies  or  Concurrent  Quality  Assessment  (CQA) 
rather  than  from  national  programs  to  combat  particular  problems  or 
change  particular  treatment  and  diagnostic  modes.    There  may  be  few, 
if  any,  specific  diagnoses  that  are  the  subjects  of  MCE  studies  or 
intensive  efforts  within  CQA  in  a  majority  of  hospitals  under  PSRO 
review.    Under  these  circumstances,  an  evaluation  of  quality  of  care 
data  on  selected  diagnoses  from  a  national  sample  may  not  reveal 
significant  changes  in  diagnostic  and  treatment  modes,  simply 
because  an  insufficient  number  of  hospitals  made  efforts  to  change 
diagnostic  and  treatment  modes  for  the  selected  diagnoses. 
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An  alternative  to  collecting  quality  of  care  data  from  a  special 
national  sample  of  hospital  patient  records  is  to  collect  data  as  a 
part  of  ongoing  efforts  toward  improvement  of  the  quality  of  care 
for  specific  diagnoses,  e.g.,  MCE  studies.    Alternatives  using  a 
national  sample  approach  and  the  ongoing  data  approach  are  described 
in  Section  3. 

Cost  Data 

Baseline  requirements  for  cost  data  include  data  on  both  cost  of 
medical  care  review  and  on  cost  of  medical  care.    A  separate  study 
has  been  initiated  which,  using  a  sample  of  30-40  hospitals,  seeks  to 
determine  cost  of  medical  care  review  in  non-PSRO  areas.    Data  will  be 
collected  for  the  periods  both  before  and  after  implementation  of  the 
new  U.R.  regulations.    The  results  of  this  study  should  satisfy  base- 
line data  requirements  for  cost  of  medical  care  review. 

Several  of  the  baseline  data  options  described  below  provide  cost  of 
care  data.    If  none  of  these  options  are  implemented,  cost  of  care 
data  for  Medicare  beneficiaries  could  be  obtained  from  SSA. 

Population  Data 

Baseline  data  on  the  number  of  residents  in  PSRO  areas  and  the  number 
of  Medicare  and  Medicaid  eligibles  are  required  for  the  computation  of 
admission  rates  and  possibly  other  population  based  measures.    Data  on 
both  overall  population  and  Medicare  eligibles  are  readily  available 
for  PSRO  areas.    Data  on  Medicaid  eligibles  may  not  be  available  for 
all  states.    In  some  of  the  states  where  the  data  are  available,  they  may 
not  be  very  accurate,  in  part  because  of  changes  in  Medicaid  eligibility. 
This  may  reduce  somewhat  the  number  of  PSRO  areas  in  which  baseline 
Medicaid  admission  rates  can  be  computed.    Since  there  is  only  one 
option  available  for  the  collection  of  this  data,  it  is  not  considered 
a  part  of  any  of  the  specific  baseline  data  options  and  is,  therefore, 
not  explicitly  included  in  the  discussion  of  the  options,  below. 


Ill .    Baseline  Data  Options 

The  options  considered  fall  in  two  classifications:  those  using  exist- 
ing or  soon  to  be  existing  (options  3,  4  and  5)  data  systems  or  those 
requiring  data  to  be  abstracted  from  individual  patient  records 
(Options  1,  2,  6  and  7).    These  options  are: 

1.  Sample  patient  records  from  national  sample  of  hospitals. 

2.  Have  individual  PSROs  sample  patient  records  from  national 
sample  of  hospitals. 


142 


3.  Use  of  abstract  service  data  (PAS  along  with  other  private 
data  systems  where  available). 

4.  Use  of  MA DOC  data. 

5.  Use  of  UHDDS  data. 

6.  Collect  retrospective  data  on  quality  of  care  as  a  part  of 
national  medical  care  evaluation  studies. 

7.  Have  individual  PSROs  sample  patient  records  prior  to 
implementation  of  particular  quality  assurance  efforts. 

In  addition,  two  or  more  of  these  options  can  be  combined  to  satisfy 
baseline  data  requirements.  In  fact,  Options  6  and  7  should  only  be 
considered  if  they  were  used  with  one  or  more  of  the  other  options. 

Option  1 :    Sample  patient  records  from  a  national  sample  of 
hospitals . 

Under  this  option,  teams  would  visit  individual  hospitals  and  abstract 
data  from  medical  records.    The  hospitals  would  be  part  of  a  national 
representative  sample  of  hospitals,  with  the  sample  stratified  by 
hospital  size,  region,  population  density  and  teaching  affiliation. 
The  sample  size  required  to  be  able  to  ascertain  with  95  percent 
confidence  that  an  observed  change  in  LOS  of  .2  days,  for  each  of  7 
common  diagnoses,  is  not  due  to  chance  is  at  least  30,000.    If  this 
level  of  accuracy  were  desired  for  each  of  4  regions,  the  required 
sample  size  is  approximately  130,000.    (See  Attachment  1  for  a  dis- 
cussion of  required  sample  size.)    These  medical  records  of  Medicube* 
beneficiaries  would  be  selected  from  records  in  a  sample  of  between 
300  and  1,000  hospitals. 

Access i bi 1 ity 

Accessibility  of  the  data  is  a  significant  problem  with  this  option. 
Given  heightened  sensitivity  to  confidentiality  and  invasion  of 
privacy  in  recent  years,  many  hospitals  selected  as  part  of  the  sample 
may  not  wish  to  provide  access  to  patients'  medical  records.    At  a 
minimum,  the  medical  staff  would  have  to  approve  granting  outside 
access  to  medical  records,  and  in  some  hospitals,  approval  of  the 
hospital  board  of  directors  may  also  be  required.    It  can  be  anticipated 
that  the  process  of  gaining  access  to  medical  records  rooms  will  be  a 
difficult  and  time  consuming  one  requiring  extensive  communication  with 


*  Medicare,  Medicaid,  and  Maternal  and  Child  Health 
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local  medical  societies  and  hospital  associations.    In  some  areas,  where 
an  anti-PSRO  attitude  prevails  within  the  medical  community,  access  may 
be  denied.    Furthermore,  requests  for  detailed  information  on  prices 
and  individual  patient  charges  may  not  allay  suspicians  that  the 
Federal  Government  is  looking  for  instances  of  overcharging  or 
fraud.    This  was  evident  from  the  GEOMET  study,  in  which  patient 
record  data  were  collected  in  six  Washington,  D.  C.  area  hospitals.  1/ 

The  accuracy  of  the  data  is  likely  to  be  relatively  good.  Primary 
sources  of  error  are  expected  to  be  omissions  and  sloppiness  in  data 
entered  in  the  medical  record  (which  is  a  source  of  error  for  all 
options)  and  abstracting  error,  in  part  related  to  unfamil iari ty  with 
notation  and  coding  methods  used  in  particular  hospitals.    Data  are 
not  available  on  error  rates  for  data  abstracted  from  medical  records 
by  outside  abstractors.    A  Hawaii  EMCRO  reabstracting  project,  in  which 
data  were  originally  abstracted  by  hospital  employees,  found  error 
rates  for  primary  diagnoses  of  between  5  and  10  percent.  ±/ 

A  problem  common  to  all  options  is  differences  among  physicians  in 
method  of  determining  the  primary  diagnosis,  especially  for  Medicare 
patients  for  whom  multiple  diagnoses  are  common.    Physicians  may 
commonly  interpret  primary  diagnosis  as  that  which  is  most  life 
threatening,  that  which  is  responsible  for  the  admission  or  that  which 
primarily  determines  the  length-of-stay. 

Adequacy 

Data  from  medical  records  are  more  detailed  and  complete  than  other 
potential  data  sources.    Detailed  information  would  be  available  on 
length-of-stay,  primary  and  secondary  diagnosis  and  the  pattern  of 
medical  care  provided  for  selected  specific  diagnoses.    An  example  of 
a  set  of  diagnostic  specific  abstract  forms  appears  as  Attachment  2. 
For  each  diagnosis,  information  would  be  collected  on  diagnostic  and 
treatment  procedures  performed,  indications  for  discharge  and  complica- 
tions that  may  extend  length-of-stay.    However,  as  noted  above,  quality 
of  care  data  collected  for  a  national  survey,  including  areas  where 
efforts  to  change  patterns  of  medical  practice  have  not  been  made  for 
the  diagnoses  chosen,  may  not  be  adequate  for  a  determination  of  the 
impact  of  PSRO  on  quality  and  patterns  of  care. 


1/  GEOMET,  working  under  government  contract  (HSA-105-74-7) ,  completed 
a  pilot  study  to  determine  the  feasibility  of  collecting  baseline 
data,  using  patient  records  as  the  source,  from  a  National  sample  of 
hospitals. 


2/  Arthur  D.  Little,  Inc.,  "Evaluation  of  Hawaii  EMCRO,"  Contract 
HSM  110-73-526,  p.  66. 
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Cost  and  charge  data  are  not  available  via  this  option,  but 

could  be  acquired  for  Medicare  and  Medicaid  beneficiaries  from  SSA 

and  state  Medicaid  agencies,  respectively. 


Time! iness 

The  data  would  be  completely  adequate  from  a  time  perspective, 
serving  as  baseline  data  for  both  already  functioning  and  future 
operating  PSROs. 

Cost 

The  cost  of  collecting  patient  record  data  is  considerably  greater 
than  for  other  options.    As  a  rough  cut  at  cost  estimation,  assume 
a  desired  sample  of  30,000  records  in  300  hospitals  in  30  PSR0  areas, 
with  an  abstractor  able  to  abstract  12-15  records  per  day.  This 
would  require  approximately  2,400  man  days  of  abstracting  time,  (10  man 
years),  plus  a  considerable  amount  of  training  and  travel  time  to 
select  the  appropriate  sample  from  each  hospital's  medical  records. 
The  personnel  cost  of  abstracting  the  data,  including  overhead,  sample 
selection,  travel  and  per  diem,  may  exceed  750,000  dollars.    This  would 
be  in  addition  to  public  relations  costs  associated  with  the  effort, 
data  processing,  analysis  and  other  costs  that  could  bring  the  total 
cost  of  the  baseline  data  study  to  1.2  million  dollars.    If  the  required 
sample  size  were  130,000  medical  records,  the  cost  of  the  study  could 
exceed  3  million  dollars. 

Option  2:    Have  individual  PSROs  sample  patient  records. 

This  option  is  similar  and  has  many  of  the  same  attributes  as  Option  1. 
Each  PSR0  that  is  included  in  a  national  sample  of  PSROs  would  hire  and 
train  personnel  to  abstract  data  from  a  sample  of  medical  records. 
One  important  distinction  between  this  and  Option  1  is  that  access  to 
data  is  likely  to  be  substantially  improved.  The  PSR0  could  be  expected 
to  have  a  continuing  operating  relationship  with  hospitals  in  its  area 
and,  presumably,  would  be  less  likely  to  experience  problems  with 
access  to  medical  records,  both  in  terms  of  denial  of  access  and 
abstracting  at  inconvenient  times. 

On  the  negative  side,  the  accuracy  of  the  data  would  be  less  than 
of  Option  1.    The  uniformity  of  the  abstracted  data  across  PSROs  will 
be  decreased  because  of  variation  in  instruction  and  training  received 
by  abstractors.    Both  the  cost  and  timeliness  of  data  under  this 
option  can  be  expected  to  be  similar  to  that  of  Option  1. 
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Option  3:    Use  Abstract  Service  data  (PAS,  along  with  other 
private  data  systems  where  avail  able)" 

The  professional  Activity  Study  (PAS)  system  is  a  data  tabulation 
service  in  which  hospitals  with  42  percent  of  total  short  term  hos- 
pital discharges  are  enrolled.    Data  abstracted  by  individual  hospitals 
and  forwarded  to  PAS  for  processing  and  tabulation  include  basic 
patient  characteristics,  length-of-stay ,  primary  diagnosis,  surgical 
procedures  performed,  expected  payment  method,  disposition  of  patient, 
basic  diagnostic  information  and  indication  whether  certain  patholog- 
ical and  radiological  tests  were  performed.    (The  standard  PAS 
abstract  form  is  included  as  Attachment  3). 

In  addition  to  PAS,  Blue  Cross  sponsored  Health  Data  Systems  (HDS) 
and  several  state  organized  hospital  data  abstract  services  process 
and  tabulate  hospital  data  very  similar  to  that  described  above. 

The  option  involves  collecting  data  either  directly  from  the  data 
tabulation  service  or  from  individual  hospitals  for  several 
diagnoses  from  a  national  sample  of  hospitals.    The  number  of 
diagnoses  for  which  data  would  be  collected,  and  the  stratification 
of  the  hospital  sample  would  be  as  described  in  Option  1.  (The 
number  of  admissions  for  which  data  would  be  collected  may  be  larger 
if  the  marginal  cost  of  increasing  the  sample  size  is  small). 

Access ibi 1 i ty 

Access  to  data  is  likely  to  present  some  problem  but  less  so  than  for 
Option  1.    Individual  hospitals  would  have  to  either  provide  the  data 
to  us  or  grant  permission  to  the  abstracting  service  to  provide  the 
data,  but  hospitals  are  not  likely  to  be  as  sensitive  concerning 
release  of  these  already  tabulated  data  as  with  outsiders  examining 
actual  patient  records. 

Accuracy  U 

The  level  of  accuracy  of  the  abstract  service  data  may  be  less  than 
for  Option  1.    Error  rates  may  be  relatively  high  due  to  abstracting 
and  coding  errors  in  preparing  the  data  for  the  data  tabulation 
services . 


3/    The  Institute  of  Medicine  of  the  National  Academy  of  Science  is 
under  government    contract  to  assess  whether  utilization  data 
obtained  from  discharge  abstract  services  can  be  used  for  base- 
line data  in  the  PSRO  program  evaluation.    Results  from  this 
study  are  expected  by  Jan.  1976. 
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Data  on  error  rates  for  abstract  service    data  are  scarce.    PAS,  the 
major  abstracting  service,  has  done  little  or  no  reabstracting  of 
hospital  records  to  test  levels  of  accuracy.    Some  inference  on 
expected  error  rates  may  be  made  from  two  studies:    1974  analysis  of 
HUP  Hospitalization  Utilization  Project  (HUP  -  a  Pennsylvania  based 
hospital  abstract  service),  and  a  study  of  error  rates  from  the  UHDD 
Demonstration  Project.    4/    The  methodology  of  the  HUP  study  is  not 
specified  but,  as  it  concerns  primary  diagnosis,  designated  errors 
would  probably  include  only  coding  errors  and  gross  diagnostic  errors 
(i.e.,  the  diagnosis  indicated  was  not  related  or  close  to  any  of 
the  diagnoses  appearing  in  the  medical  record).    The  average  error 
rate  was  only  .15  percent  during  the  July-December  1973  period.  The 
error  rate  for  any  specific  abstract  item  never  exceeded  1/2  of  one 
percent  and  only  3  percent  of  individual  records  have  any  errors. 
The  UHDD  error  rate  for  principal  diagnosis  was  10  percent,  but  the 
methodology  included  in  depth  reabstracting  of  the  medical  records. 
The  error  rate  includes,  in  addition  to  coding  errors,  errors  in 
determining  which  of  several  diagnoses  appearing  in  the  medical 
record  should  be  classified  as  principal,  and  designating  the 
appropriate  diagnostic  code  even  where  "two  codes  describe  virtually 
identical  conditions."  5/    Thus,  the  error  rate,  when  measured  in  this 
way,  may  not  be  much  less  than  10  percent  even  under  option  1, 
where  baseline  data  would  be  abstracted  directly  from  the  medical 
records.    An  overall  assessment  of  error  rates  from  the  above 
sources  may  lead  one  to  conclude  that  the  marginal  increase  in  error 
rates  is  not  a  substantial  problem. 

Discussions  with  researchers  in  the  health  care  delivery  area, 
however,  lead  one  to  be  less  sanguine  concerning  abstract  review 
data  reliability.    While  most  abstract  services  will  pick  up  obvious 
inconsistencies  (i.e.,  male  pregnancies),  there  are  numerous 
possibilities  for  other  coding  errors  that  would  not  be  discovered. 
Because  the  incidence  of  these  type  errors  is  unknown,  some 
researchers  are  wary  about  relying  on  abstract  service  data  to  satisfy 
PSRO  baseline  data  requirements.    Before  abstract  service  data  would 
be  used,  a  reabstracting  study  of  the  major  abstracting  services  data 
to  determine  error  rates  would  be  performed.    Planning  and  initial 
arrangements  for  such  a  study  have  been  initiated. 

An  additional  problem  is  that  medical  records  librarians  and  coding 
clerks  in  different  hospitals  use  different  guidelines  and 


4/    "Focus,"  Hospital  Utilization  Project,  November-December  1974; 
"Uniform  Hospital  Discharge  Data  Demonstration  -  Summary 
Report,"  BHSR  and  E,  NCHSR  and  E,  PHS  Grant  No.  500400. 

5/    "Uniform  Hospital  Discharge  Data  Demonstration,"  p.  39. 
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criteria  in  determining  primary  diagnosis,  in  listing  other 
diagnoses,  etc.    This  may  even  vary  over  time  within  the  same 
hospital  as  personnel  change. 

There  may  be  a  problem  of  hospitals  participating  in  abstract  services 
not  being  representative  of  all  U.S.  hospitals.  Participation  in  private 
abstract  services  is  voluntary  and  hospitals  that  do  participate  may 
be  more  concerned  with  collecting  data  useful  for  utilization  review 
than  hospitals  that  do  not  participate.    There  is  no  simple  way 
to  check  this.    What  is  known  is  that  for  PAS,  the  largest  abstract 
service,  a  far  greater  proportion  of  hospitals  of  more  than  100  beds 
(39  percent  are  covered  by  PAS  than  of  hospitals  of  less  than  100 
beds  (13  percent.)  6/    It  may  be  that  PAS  hospitals,  as  a  group,  have 
superior  medical  records  departments  than  hospitals  that  don't 
participate. 

Adequacy 

Abstract  service  data  would  be  adequate  for  number  of  admissions 
and  LOS  by  diagnosis,  but  because  data  on  diagnostic  and  treatment 
procedures  are  relatively  scant,  the  data  would  not  suffice  as  base- 
line data  for  quality  of  care.    As  indicated  earlier,  however, 
Options  1  and  2  may  not  be  adequate  for  this  purpose  either. 

Very  few  hospitals  report  cost  data  or  charge  information,  which  is 
optional  in  PAS.    Medicare  and  Medicaid  charge  and  cost  data  would 
have  to  be  used  instead.    These  data,  while  available  by  principal 
diagnosis,  would  not  be  available  for  the  PAS  sample  of  admissions. 

Timeliness 

The  data  would  be  adequate  from  a  time  perspective.  Approximately 
90  percent  of  PAS  data  are  available  within  three  months  of  the 
end  of  the  period  within  which  hospital  discharges  occurred,  and 
data  would  be  available  before  PSROs  become  (or  became)  operational 
in  particular  areas. 

Cost 

The  cost  of  Option  3  is  considerably  less  than  of  options  1  and  2. 
The  cost  of  data  acquisition  should  be  in  the  range  of  that  charged 
individual  hospitals,  which  is  30  cents  per  admission.    There  would 
be  additional  costs  of  communicating  with  individual  hospitals  for 
purposes  of  their  granting  0PSR  access  to  their  data,  either 

6/    Commission  on  Professional  and  Hospital  Activities,  Length  of 
Stay  in  PAS  Hospitals,  1974,  Table  A. 
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directly  or  through  abstracting  services. 
Option  4:    Use  of  MADOC  data. 

The  Social  Security  Administration  regularly  publishes  data  abstracte 
from  Medicare  claims  forms,  based  on  a  20  percent  sample  of  those 
forms.    A  listing  of  the  data  elements  collected  for  the  Medicare 
Analysis  of  Days  of  Care  (MADOC)  is  enclosed  as  Attachment  4.  Inform* 
tion  is  available  concerning  length-of-stay,  discharge  diagnosis,  the 
performance  of  surgical  procedures  (yes  or  no),  the  existence  of 
secondary  diagnoses  and  charges  for  nine  hospital  charge  centers. 
Periodic  reports  are  made  available  to  all  hospitals  within  each  275 
hospital  areas,  which  provide  data  for  each  hospital  in  the  area.  Thi; 
enables  individual  hospitals  to  compare  their  performance  with  that 
of  similar  hospitals  in  their  immediate  area.    (Estimates  of  expected 
length-of-stay,  based  on  statistical  analysis  of  hospital  and 
patient  characteristics,  are  also  provided.) 

In  the  275  area  format  currently  available,  the  MADOC  data  are  clearly 
inappropriate  for  use  as  a  primary  source  of  baseline  data  for  the 
PSRO  program.    However,  plans  exist  to  change  area  designations  to 
coincide  with  designated  PSRO  areas. 

Accessibility 

The  data  are  accessible.    The  system  for  abstracting  and  publishing 
the  data  for  individual  hospitals  is  already  operational. 

Accuracy 

Error  rates  are  probably  similar  to  those  associated  with  abstract 
service  data.    Primary  sources  of  error  for  primary  diagnosis  include 
sloppiness  in  entering  information  in  the  medical  record,  and  errors 
(judgment  and  coding)  in  abstracting  data  for  the  Medicare  claims 

form. 

Adequacy 

MADOC  data  are  incomplete  as  a  source  of  baseline  data.    Data  are  not 
included  for  Medicaid  beneficiaries,  who  constitute  over  one  third 
of  Medicube  admissions.    Additional  problems  are  that  discharge 
diagnosis  is  used  rather  than  primary  diagnosis  or  admitting  diagnosis, 
and  information  is  not  abstracted  (and  apparently,  does  not  exist  for 
a  significant  proportion  of  claim  forms  submitted  to  Medicare)  on 
secondary  diagnoses  and  surgical  procedures  performed.  Significant 
changes  in  data  submission  requirements  and  in  the  claims  form  would 
be  required  to  correct  these  latter  problems. 
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Time! iness 

The  timeliness  of  MADOC  data  may  pose  a  problem.    While  data  would  be 
available  for  both  new  and  established  PSROs,  there  have  been  lags 
in  excess  of  two  years  between  the  end  of  the  data  collection  period 
and  when  the  data  are  available  for  use.    Hopefully,  this  period 
could  be  reduced  to  something  less  than  one  year.    (SSA  has  indicated 
that  they  could  provide  us  with  1973  data  for  a  sample  of  PSRO  -  non- 
PSRO  areas  sometime  in  the  summer  of  1975.) 

Cost 

The  cost  of  this  option  is  minimal  as  the  data  collection  and  prepara- 
tion system  is  already  in  place. 

Option  5:    Use  of  UHDDS  Data 

It  is  anticipated  that  by  early  1976,  all  hospitals  will  be  required 
to  collect  a  Uniform  Hospital  Discharge  Data  Set  for  each  Medicare 
and  Medicaid  admission. 

The  UHDDS  data  includes  information  on  basic  patient  characteristics, 
length-of-stay ,  principal  and  secondary  diagnoses,  and  significant 
procedures.    For  Medicare  admissions,  the  UHDDS  will  be  combined  with 
the  claims  form  so  that  cost-charge  information  would  also  be  available. 
The  UHDDS  appears  in  Attachment  5. 

Access ibi 1 ity 

There  are  no  confidentiality  associated  access  problems  with  Option  5 
as  the  UHDDS  information  will  be  forwarded  to  HEW,  although  the  route 
is,  as  of  yet,  undetermined. 

Accuracy 

The  accuracy  of  the  data  can  be  expected  to  be  equal  to  or  superior 
to  that  of  Option  3  (abstract  service).    The  error  rate  of  10 
percent  for  principal  diagnosis  noted  earlier  from  the  UHDDS 
demonstration  project  is  applicable  to  this  option.    The  area  where 
this  option  can  be  expected  to  be  superior  to  Option  3  is  in 
comparability  across  hospitals.    Instruction  and  to  the  extent  that 
it  occurs,  training  for  completing  the  UHDDS  will  be  uniform  across 
hospitals,  making  inter-hospital  comparison  more  reliable  than  with 
Option  3. 
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Adequacy 

The  UHDDS  data  can  be  expected  to  be  adequate  for  utilization  data. 
While  UHDDS  does  not  contain  cost  or  charge  information,  charge  data 
for  Medicare  beneficiaries  in  the  UHDDS  baseline  data  sample  should 
be  available  (from  SSA)  and  should  be  useful  in  assessing  the  impact 
of  PSRO  on  cost  of  care.    Only  limited  information  will  be  available 
from  UHDDS  with  respect  to  quality  data.    Therefore,  it  will  not 
be  adequate  as  baseline  data  for  assessment  of  quality  of  care. 

Time! iness 

There  may  be  a  UHDDS  problem  with  the  timely  availability  of  the 
UHDDS  data.    Implementation  of  UHDDS  will  not  start  before  calendar 
year  1976.    These  data  would  not  be  adequate  baseline  for  original, 
newly  conditional,  (FY75),  and  other  PSROs  that  become  conditional 
in  calendar  year  1975.    For  these  PSROs,  data  would  have  to  be 
obtained  from  one  of  the  other  options. 

Cost 

The  cost  of  acquiring  baseline  data  under  Option  5  should  be  substant- 
ially less  than  Options  1,  2  or  3.    All  abstracting  costs  and  much  of 
the  computer  cost  will  have  been  incurred  as  part  of  the  monitoring 
functions  of  the  Medicare,  Medicaid  and  PSRO  programs.  Marginal 
costs  associated  with  processing  of  baseline  data  for  this  option 
can  be  expected  to  be  minimal. 

Option  6:    Collect  retrospective  quality  of  care  data  as  part  of 
National  Medical  Care  Evaluation  studies. 

(Both  this  and  the  following  option  would  provide  baseline  data 
primarily  on  quality  of  care  and  would  be  expected  to  be  used  along 
with  some  other  option(s)  which  would  provide  utilization  and  cost 
of  care  data. ) 

Under  this  option,  OPSR-BQA  would  sponsor  a  series  of  diagnostic 
specific  studies  in  hospitals  in  which  efforts  were  made  to  improve 
the  level  of  care  for  these  diagnoses     The  purpose  would  be  to 
determine  if  specific  PSRO  sponsored  efforts  to  improve  quality  of 
care  achieved  their  goals.    These  studies  would  be  initiated  during 
the  second  and  perhaps  third  year  of  the  operational  life  of  the 
PSRO.    As  an  integral  part  of  these  studies,  retrospective  data  would 
be  collected  for  the  period  prior  to  PSRO  operational  status,  with 
the  primary  source  of  these  data  expected  to  be  medical  records.  7/ 


]_/  The  implementation  of  this  option  would  require  a  major  policy 
change  in  the  PSRO  program. 
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Access,  Accuracy  and  Adequacy 

No  serious  access  problems  are  anticipated  with  this  option.  It 
should  provide  accurate  and  adequate  baseline  data  on  quality  of 
care.    It  is  superior  to  Options  1  and  2  (sample  medical  records  from 
a  national  sample)  in  that  it  would  not  seek  to  determine  if  changes 
occurred  in  diagnostic  and  treatment  modes  for  diagnoses  for  which 
no  PSRO  effort  was  made  to  produce  change.    It  is  therefore  a 
superior  test  of  the  impact  of  PSRO  on  quality  of  care,  where  PSRO 
sought  to  have  an  impact. 

A  defect  of  this  option  may  be  that  it  could  subject  the  evaluation 
effort  to  charges  that  those  hospitals  chosen  for  study  were  not  a 
random  selection  of  hospitals  in  PSRO  areas,  but  those  whose  efforts 
to  improve  quality  of  care  were,  a  priori ,  likely  to  be  most 
successful.    This  defect  could  be  minimized  by  developing  a  method- 
ology for  randomly  selecting  hospitals  for  study  among  all  PSRO 
hospitals  in  which  efforts  were  made  to  improve  care  for  specific 
diagnoses.    This  is  desirable  and  feasible. 

Timeliness  and  Cost 

The  timeliness  of  the  data  availability  is  not  likely  to  pose  any 
problems.    The  cost  of  this  option  is  unknown,  but  probably  less  than 
that  of  Options  1  and  2. 

Option  7:    Have  individual  PSROs  sample  patient  records  prior 
to  implementation  of  particular  quality  assurance 
efforts . 

Under  this  option,  whenever  a  PSRO  undertakes  an  MCE  study  or  an  effort 
to  establish  criteria  and  standards  for  treating  particular  conditions, 
an  integral  part  of  that  effort  would  be  a  systematic  evaluation, 
using  all  relevant  hospital  records,  of  all  aspects  of  current 
diagnostic  and  treatment  modes  for  that  condition. 

Access,  Accuracy  and  Adequacy 

No  access  problems  are  anticipated  for  this  option.    Both  the  accuracy 
and  adequacy  of  baseline  data  under  this  option  should  exceed  that 
of  the  other  options  for  the  specific  purpose  noted  above.  Specific 
instructions  and  guidelines  on  method  of  obtaining  baseline  data 
would  be  provided  by  OPSR-BQA  (probably,  after  development  by  an 
outside  contractor),  and  resources  would  be  available  within 
OPSR-BQA  to  assist  in  the  data  collection  effort.    Because  each  data 
collection  effort  would  be  tailored  to  and  part  of  specific  efforts 
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to  change  modes  of  practice,  it  would  be  more  suitable  and  productive 
for  measuring  PSROs '  impact  on  modes  of  practice  and  outcome  than 
Options  1  or  2. 

Timeliness  and  Cost 

The  data  are  expected  to  be  available  on  a  timely  basis.    The  cost  of 
collecting  baseline  data  under  Option  7  may  be  substantial,  but  much  it 
would  generally  be  incurred  as  part  of  MCE  studies  and  adherance  to 
quality  standards  efforts.    The  marginal  cost  of  systematizing  the 
data  collection  effort  so  that  the  data  can  be  used  as  baseline 
data  on  process  and  outcome  may  not  be  large  and  should  be  considerably 
less  than  of  data  collection  under  Option  1  or  2. 


IV.    Collecting  baseline  data  from  multiple  sources 

It  was  explicitly  noted  that  use  of  either  Option  6  or  7  requires  the 
use  of  an  additional  option  for  utilization  and  cost  of  care  baseline 
data.    It  may  also  be  desirable  to  jointly  use  other  options  for 
collection  of  utilization  and  cost  of  care  data.    MA DOC  data  could  be 
used  to  supplement  UHDDS  in  areas  where  implementation  of  UHDDS  is  too 
late  to  serve  the  baseline  purpose.    Similarly,  if  abstracting 
service  were  the  primary  baseline  data  option,  MADOC  or  UHDDS  might 
be  used  as  a  supplement  in  PSRO  areas  where  PAS  or  other  available 
data  tabulation  services  are  not  in  common  use,  or  where  access  to 
the  data  was  not  granted  by  hospitals. 

The  cost  of  this  flexibility  may  be  minimal  as  several  of  the  data 
sources  considered  for  use  as  baseline  data  will  be  available  whether 
used  for  this  purpose  or  not. 

The  extent  to  which  each  of  the  seven  baseline  data  options  meet  the 
criteria  for  access,  accuracy,  adequacy,  timeliness  and  cost  are 
noted  in  summary  form,  in  Table  1. 
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Attachment  1  -  Sample  Size  Analysis 


The  following  sample  size  analysis  has  been  carried  out  to  minimize 
the  resources  necessary  to  compute  PSRO  baseline  data  on  the  length 
of  stay  for  Medicube  hospital  admissions.    Tables  have  been  pre- 
pared for  two  alternative  data  collections  of  hospital  medical  records 
of  length  of  stay  by  diagnosis.    Table  1  describes  the  required 
sample  sizes  for  95%  confidence  intervals  with  tolerances  of  one 
quarter  day  (  +  1/4  day)  for  each  of  the  four  regions  of  the  nation  by 
each  of  seven  key  diagnoses.    Using  the  sample  sizes  from  Table  1, 
it  will  be  possible  to  measure  and  access  changes  in  the  mean 
length  of  stay  as  small  as  .18  days  for  a  particular  diagnosis  in  a 
particular  region  of  the  country.    Table  2  describes  the  required 
sample  sizes  for  95%  national  confidence  intervals  with  tolerances 
of  one-quarter  day  (+  1/4  day)  for  each  of  seven  key  diagnoses. 
Using  the  data  from  Table  2,  it  will  be  possible  to  measure  and 
access  changes  in  the  national  mean  length  of  stay  as  small  as  .18 
days  for  each  of  the  diagnosis. 

The  source  data  for  this  analysis  came  from  the  Social  Security 
Administration  publication  of  "Length  of  Stay  by  Diagnosis"  for 
1969  Medicare  admissions  and  from  the  Commission  on  Professional 
and  Hospital  activities  publication  of  "Length  of  Stay  in  PAS 
Hospitals"  for  1972  admissions.    The  limitations  of  this  data  arise 
because  there  are  variances  between  PAS  and  SSA  data  due  to  the 
difference  in  base  year,  differences  in  the  computation  of  length 
of  stay  (LOS),  and  differences  in  the  definition  of  short-term 
hospitals.  PAS  data  are  for  1972  while  SSA  data  are  for  1969.  PAS 
data  excludes  patients  who  died,  were  transferred  to  other  hospitals, 
were  discharged  against  medical  advice,  or  stayed  100  days  or 
longer.    SSA  data  excludes  those  cases  for  which  the  hospital 
stay  was  over  90  days.    PAS  uses  the  hospital  adoptation  of  ICDA  codes 
(H-ICDA)  while  SSA  uses  the  seventh  revision  of  ICDA  codes  (ICDA-7). 

PAS  includes  only  non-federal  hospitals  with  an  average  stay  of 
less  than  45  days  while  SSA  includes  only  non-federal  hospitals 
with  an  average  stay  of  less  than  30  days.    None  of  these  differences 
in  the  PAS  and  SSA  data  preclude  their  use  to  compute  variance 
estimates  for  the  various  diagnoses.    However,  assumptions  must  be 
made  that  the  processes  which  generate  LOS  data  are  stable  with 
respect  to  their  variability.    There  is  no  need  to  assume  that  the 
average  LOS  is  equal  for  both  sources  of  data  or  constant  over 
time. 

The  first  step  in  the  analysis  was  to  compute  a  weighted  variance 
for  the  length  of  stay  in  each  region  using  the  SSA  and  PAS  data. 
Table  1  was  then  prepared  for  each  region  without  assuming  any 
limitations  on  the  total  number  admissions  for  any  diagnosis. 
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The  analysis  for  Table  2  began  with  the  projection  of  the 
percentage  of  SSA  and  PAS  admissions  for  each  diagnosis  to  compute 
the  national  expected  number  of  hospital  admissions  for  a  quarter 
when  the  annual  rate  of  Medicare  admissions  is  seven  million  and  the 
annual  rate  of  Medicaid  admissions  is  four  million.    Then  the 
expected  total  number  of  admissions  during  a  quarter  for  each 
diagnosis  was  divided  into  the  expected  number  of  admissions  in 
each  of  the  four  regions  of  the  nation  in  the  same  proportion  as 
was  observed  in  the  source  data.    Each  of  these  four  regions  was  then 
considered  as  a  strata  and  the  optimal  sample  size  for  each  region 
was  computed  by  diagnosis.    These  sample  sizes  are  optimal  in  that 
they  will  produce  95%  confidence  intervals  for  national  mean  with 
a  tolerance  of  one-quarter  day  for  each  diagnosis  using  the  absolute 
minimum  number  of  observations  of  medical  records. 

While  the  assignment  of  the  samples  is  presently  only  for  region 
by  diagnosis,  it  is  possible  to  further  subdivide  the  assignment  to 
measure  length  of  stay  by  hospital  size,  proprietorship,  training 
affiliations,  or  other  hospital  characteristics. 
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Attachment  3 


168. 


THE  PAS  SYSTEM 

1973  CASE  ABSTRACT 


PATIENT  MUMWI 

EBEBSEI 

discharge  o*re  ■, 

■mm 

73 

*i 

73 

i 

™_. 

ADMISSION  DATE 


EXPECTED  PATMENT 

2 


.  nowitm  XX  ^  ^ 
Crou    A^    Nuwfifa    _ 


14  EXAMS       ,    l5CA8f  UNITS 

I    !,  I 


1  '  I 

A3  TESTS  AND  X-RATS  ' 


I 


34  ADMISSION  HG6  OB  MCT  35 


]7 


21  WEIGHT  22  HEIGHT 

■  "IHtRAFl         ■  i 

'3      A  DRUGS  „ 


<iirhv.o; 

Steroid 


^■p'lAnl.coog-  ^ 
4j*!0*.r."«  ^ 


^-Hf^>  A 

..II.I..M..   ■    SUUGEHr     ,         i  ■- 


Vi 


'! 


37  URINE  TESTS  -'EC  . 


3r 


no  path  report 


S3  ADMISSION 
6LOO0 
PRESSURE 


52  COHSUlUNONi 

Given  8t 


i 


SURGEON  A  I 
~rrj£'£'.=:  T     I       SURGEON  B 


OPERATIONS 

t    '.J^^r  ,-/^-(dJ       °F  SURGEON  0 

[  i^lfelb  if  !  X 


33      TIME  OF  EARLIEST  SURGERT 
(Oft  DELIVERY  if  bafo<*  iwrgtn 


i 


TEV,  JfpATU^E 


38  HISTOLOGY 


uOOD  C H t  M i  .  '•  ■ 


/ 


PAS  fo.m  700-79  P. .mad  in  U  I  A. 
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Attachment  4 


169- 


Record  Format— MADOC-5  (1/72-6/72) 


Item 


Number  of 
Positions 


Code 


Field 
Position 


1. 

HI  claim  number 

11 

Actual  number 

1-11 

2. 

Provider  number 

6 

Actual  number 

12-17 

3. 

Intermediary  number 

5 

Actual  number 

18-22 

4. 

Hospital  service  area  number 

3 

Actual  number 

23-25 

5. 

Discharge  date 

6 

6  digit  number,  year,  month, 
day 

26-31 

6. 

Discharge  diagnosis  (ICDA 
code) 

4 

Actual  number 

32-35 

7. 

Length  of  stay 

3 

Date  of  discharge  minus  date 
of  admission.     If  difference 
is  0  make  it  1. 

36-38 

8. 

Age 

3 

Define  age  as  a  3  position 
number,  i.e..  066.  067.  068 

pfr.  .    AS   of               hirt-hrlav  on 

date  of  admission. 

39-il 

9. 

Sex 

1 

0  -  female,  1  -  male 

42 

10.  Race 

11.  Additional  diagnosis 

12.  Surgical  procedures 

13.  Discharge  status 

14.  Day  of  admission 


1  0  -  unknown,  1  -  white,  43 

2  -  negro,  3  -  other 

1  yes  -  1,  no  -  0  44 

1  yes  -  1,  no  -  0  45 

1  0  -  alive,  1  -  dead  46 

7  0  or  1.    A  code  1  is  to  be  4  7-53 

inserted  in  the  relative 
position  to  indicate  the 
actual  day  of  the  week;  all 
others  will  contain  0's. 
NOTE:     Let  the  first  of  the  7 
position  field  be  Sunday  and 
continue  with  each  succeeding 
day  so  that  the  7th  will  be 
Saturday. 
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Item 

Number  of 
Positions 

15. 

Intensive  care  charges 

4 

16. 

Operating  room  charges 

4 

17. 

Pharmacy  charges 

4 

18. 

Laboratory  charges 

4 

19. 

Radiology  charges 

4 

20. 

Supplies  charges 

4 

21. 

Anesthesia  charges 

4 

22. 

Inhalation  therapy  charges 

4 

23. 

Blood  charges 

4 

24. 

Total  charges 

4 

25. 

Type  of  service 

1 
1 

26.     Type  of  control  1 

1 
1 
1 
1 


27.     Number  of  facilities  and  2 
services 


2 


Field 

Code  Position 

Dollars  only  54-57 

Dollars  only  58-61 

Dollars  only  62-65 

Dollars  only  66-69 

Dollars  only  70-73 

Dollars  only  74-77 

Dollars  only  78-81 

Dollars  only  82-85 

Dollars  only  86-89 

Dollars  only  90-93 

General-short  term  9& 

RnPC^fll ^v-Qhnr^    harm  0^ 


A  code  1  is  to  be  inserted  in 
the  relative  position  to 
indicate  the  type  of  service; 
the  other  position  will 
contain  a  0.     For  long  stay 


hospitals  use  "00." 

Government  non-federal  96 

Church  97 

Proprietary  98 

Federal  99 

Other  non-profit  100 


A  code  1  is  to  be  inserted 
in  the  relative  position  to 
indicate  the  type  of  control; 
all  other  position  will 
contain  0's. 

01,  02,  03,  etc.  101-102 
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Item 

28.  Medical  school  affiliation 

29.  Bed  capacity  (adult) 

30.  Number  of  active  members  on 
medical  staff 

31.  Number  of  resident  training 
programs 

32.  Periodical  interim  payment 

33.  Date  of  Admission 

34.  Date  of  Surgery 

35.  Surgery  code  (CPT  code) 

36.  Blood  furnished  (pints) 

38.     Total  accommodation  charges 


Number  of 
Positions 

1 

4 

4 


Code 
yes  -  1,  no  -  0 
Actual  number 
Actual  number 

Actual  number 

yes  -  1,  no  -  0 

6  digit  number,  year,  month, 
day 

6  digit  number,  year,  month, 
day 

Actual  number 

2  digit  whole  pints 


Dollars  only,  lines  19B  +19C 
419D  from  SSA-1453 


11 


Field 
Position 

103 

104-107 
108-111 

112-113 

114 
115-120 

121-126 

127-130 
131-132 

136-139 
140-150 


39.  Blanks 
Final  tape  output  as  follows: 
Density:     9  Track  High  Density  1600  BPI 

Blocked:     40  claims/records  or  6000  characters.     If  less  than  40  claims  are  available, 
use  9's  padding. 

Sorted:    Major  key:     State  code  (field  positions  12-13) 
Secondary  key:     Hospital  Service  Area  number 
Minor  key:     Discharge  date 


End  of  file  mark  on  each  tape. 
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ATTACHMENT 

UNIFORM  HOSPITAL  DISCHARGE  DATA  SET  (UHDDS) 


UHDDS 

1 .      Person  Identification 

Each  admission  is  to  be  reported  by  the  patient's  unique  social 
security  number.    For  newborns  and  children  not  having  a  social 
security  number  but  covered  by  Medicaid,  the  recipient  I.D. 
number  is  to  be  used.    (A  unique  number  is  essential  to  assure 
record  linkage  for  multiple  admissions  of  the  same  individual). 

If  the  hospital  also  assigns  a  medical  record  number  which  differs 
from  the  social  security  number  of  the  recipient  I.D.  number, 
it  is  also  to  be  furnished  (to  facilitate  retrieval  of  individual 
case  records) . 

2.  Date  of  Birth 

Month,  day  and  year  of  birth 

3.  Sex 


Male,  female 

4.  Race 

White,  black,  other 

5.  Residence 
ZIP  code 

6.  Hospital  Identification 

The  provider  number  assigned  by  the  Medicare  Program  and  used 
by  Medicare  and  Medicaid  in  the  hospital  certification  process. 
NOTE:    Report  to  BQA  hospital  identification  number  assigned  and 
used  by  the  PSRO.    Additional  information  about  the  hospital, 
such  as  its  bed  size,  will  also  be  reported  according  to  BQA 
instructions  now  under  development. 

7.  Admission  Date  and  Hour 


Month,  day,  year  and  hour  of  admission 
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Discharge  Date 

Month,  day  and  year  of  discharge 
Attending  Physician 

This  is  the  physician  primarily  responsible  for  the  care  of  the 
patient  from  the  beginning  of  this  hospital  episode.    In  determin- 
ing the  physician  primarily  responsible,  the  following  criteria 
apply: 

a.      If  the  patient  has  a  private  attending  doctor  who  arranged 
for  his  admission  to  the  institution  and  directed  his  care 
therein,  this  physician  is  normally  considered  to  be  the 
attending  physician  in  the  hospital. 

b.    If  the  patient  does  not  have  a  private  doctor,  the  physician 
primarily  responsible  in  the  hospital  is  the  staff  member 
or  resident  to  whom  the  patient  is  assigned  and  for  whose 
care  he/she  is  legally  responsible. 

The  physician  is  to  be  identified  by  his/her  unique  social 
security  number. 

Operating  Physician 

This  is  the  physician  who  performed  the  prinicpal  procedure.  The 
physician  is  to  be  identified  by  his/her  unique  social  security 
number. 

.  Diagnoses  (Principal  and  Other) 
All  diagnoses  that  affect  the  current  stay. 

a.  Principal  Diagnosis  is  to  be  designated  and  is  defined  as: 
the  condition  established  after  study  to  be  chiefly  respon- 
sible for  occasioning  the  admission  of  the  patient  to  the 
hospital  for  care. 

b.  Other  Diagnoses  to  be  listed  are: 

All  conditions  that  coexist  at  the  time  of  admission,  or 
develop  subsequently,  which  affect  the  treatment  received 
and/or  the  length  of  stay.    Diagnoses  that  relate  to  an 
earlier  episode  which  have  no  bearing  on  this  hosiital 
stay  are  to  be  excluded. 

NOTE:    Only  principal  diagnosis  and  the  number  of  diagnoses 
recorded  are  necessary  for  reporting  to  BQA. 
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12.  Procedures  (Principal  and  Others) 

a.  All  procedures  performed  in  operating  rooms  are  to  be 
reported  with  the  dates.    In  addition  to  these  procedures, 
all  other   significant  procedures  are  to  be  recorded.  A 
significant  procedure  is  one  which  carries  an  operative  or 
special  facilities  or  equipment.    Some  examples  of  such 
procedures  are  cardiocatheterization,  angiography, 
endoscopy,  and  supervoltage  radiation  therapy. 

b.  When  more  than  one  procedure  is  recorded  the  principal 
procedure  is  to  be  designated.    In  determining  which  of 
several  procedures  is  the  principal,  the  following 
criteria  apply: 

(1)  The  principal  procedure  is  one  which  was  performed  for 
definitive  treatment  rather  than  one  performed  for 
diagnostic  or  exploratory  purposes,  or  was  necessary 
to  take  care  of  complication. 

(2)  The  principal  procedure  is  that  procedure  most  related 
to  the  principal  diagnosis. 

NOTE:    Only  principal  procedure,  date  and  the  number  of 
additional  procedure  recorded  are  necessary  for 
reporting  to  BQA. 

13.  Disposition  of  Patient 

a.  Transferred  to  another  short-term  general  hospital 

b.  Discharged  or  transferred  to  skilled  nursing  facility  (SNF) 

c.  Discharged  or  transferred  to  an  intermediate  care  facility 

(ICF) 

d.  Discharged  or  transferred  to  another  institution 

e.  Discharged  to  home  or  self-care  (routine  discharge) 

f.  Discharged  to  home  under  care  of  an  organized  home  health 

service 

g.  Left  against  medical  advice 

h.  Died 
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*14.  Expected  Principal  Source  of  Payment 


a. 

Sel f-pay 

b. 

Workmen's  compensation 

c. 

Medi  care 

d. 

Medi  caid 

e . 

Other  Government  Payment  (e.g.  CHAMPUS) 

f . 

Blue  Cross 

g. 

Insurance  company 

h. 

No  charge  (free 5chari ty 5  special  research,  or  teaching) 

i . 

Other  (e.g.  relatives,  friends) 
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